Measuring Effective Democracy

This article discusses methodological problems related to operationalizing substantive definitions of democracy. The article argues that index-constructors need to be particularly conscious of measurement level issues. If not, their indexes may face severe reliability and validity problems, which in turn may bias empirical analyses utilizing the indexes. The article focuses particularly on the "effective democracy" measure developed by Ronald Inglehart and Christian Welzel. The measure has been used by Inglehart and Welzel in several studies, particularly for empirically testing hypotheses deduced from their version of modernization theory. These tests have generated very strong results in favor of the theory. The article is sympathetic to Inglehart and Welzel's goal of capturing "substantive" rather than "formal" democracy, but is critical of the specific measure proposed. The measure has several unfortunate theoretical and distributional properties; the empirical scores generated by the measure are often highly misleading. Empirical analysis suggests the index is biased, and that rich, Western countries are particularly favored. Utilization of the measure in statistical analysis may therefore lead to false inferences.