Exploring Subjective and Objective Measures of System Effectiveness

The effectiveness of information systems (IS) is essential for evaluating user performance between their behavior and computer. However, the effectiveness evaluation poses challenges due to the multidimensional nature of measures, especially in evaluating business intelligence systems (BI). This study aims to investigate measures from both subjective and objective of BI effectiveness and classify BI effectiveness into five dimensions based on the IS maturity model, including Operate, Consolidate, Integrate, Optimize, and Innovate. The study compares results obtained through self-reported with computer-recorded measures to gain a comprehensive understanding of system utilization. This research received 1480 questionnaires, accompanied by the collection of 1-year log data reflecting the actual system usage of each respondent. The findings reveal a partial alignment between self-reported and actual usage frequency depending on divergent effectiveness levels. These insights contribute to the complex dynamics underlying user behavior and provide practical implications for organizations seeking to enhance effective BI utilization.