A general concept of scoring rules: general definitions, statistical inference, and empirical illustrations

We develop the general conceptual, mathematical and statistical foundations of behavioral social choice for scoring rules. Traditional scoring rules are difficult to assess empirically because one rarely observes the deterministic complete linear orders that they require as input. We provide a general concept of scoring rules in terms of a broad range of mathematical representations of preference or utility, namely arbitrary finite binary relations, probability distributions over such relations, real valued multi-criteria utility vectors and real valued random utility representations. We extend Regenwetter et al.'s (Behavioral social choice. Cambridge University Press, Cambridge, 2006) statistical framework to a more general setting. We illustrate the general modeling and statistical tools by applying them to four well known sets of survey data. We illustrate two potential problems that have previously received little attention and that deserve systematic study in the future: (1) Scoring rule outcomes can suffer from model dependence in that the social welfare functions computed from ballot, survey, or hypothetical data may depend on implicit or explicit modeling assumptions. (2) Scoring rule outcomes may suffer from low statistical confidence in that the correct assessment of social orders from empirical data can be far from certain. We also illustrate the empirical congruence among conceptually competing social choice methods.