Unbiased assessment of disease surveillance utilities: A prospect theory application

Objectives We contribute a new methodological approach to the ongoing efforts towards evaluating public health surveillance. Specifically, we apply a descriptive framework, grounded in prospect theory (PT), for the evaluation of decisions on disease surveillance deployment. We focus on two attributes of any surveillance system: timeliness, and false positive rate (FPR). Methods In a sample of 69 health professionals from a number of health related networks polled online, we elicited PT preferences, specifically respondents' attitudes towards gains, losses and probabilities (i.e., if they overweight or underweight extreme probabilities) by means of a series of lotteries for either timeliness or FPR. Moreover, we estimated willingness to pay (WTP) for improvements in the two surveillance attributes. For contextualization, we apply our framework to rabies surveillance. Results Our data reveal considerable probability weighting, both for gains and losses. In other words, respondents underestimate their chances of getting a good outcome in uncertain situations, and they overestimate their chances of bad outcomes. Moreover, there is convex utility for losses and loss aversion, that is, losses loom larger than gains of the same absolute magnitude to the respondents. We find no differences between the estimated parameters for timeliness and FPR. The median WTP is $7,250 per day gained in detection time and $30 per 1/10,000 reduction in FPR. Conclusion Our results indicate that the biases described by PT are present among public health professionals, which highlights the need to incorporate a PT framework when eliciting their preferences for surveillance systems. Author summary In this paper we contribute a new methodological approach to the ongoing efforts towards evaluating public health surveillance. Specifically, we apply a descriptive framework for the evaluation of decisions on disease surveillance deployment. We focus on two attributes of any surveillance system: timeliness and false positive rates. In a sample of 69 health professionals from a number of health related networks polled online, we elicited preferences, specifically respondents' attitudes towards gains, losses and probabilities (i.e., if they overweight or underweight extreme probabilities) by means of a series of lotteries. For contextualization, we apply our framework to rabies surveillance. Our data reveal that respondents underestimate their chances of getting a good outcome in uncertain situations, and they overestimate their chances of bad outcomes. Moreover, losses loom larger than gains of the same absolute magnitude to the respondents. We find no differences between the estimated parameters for timeliness and false positive rates. Our results indicate that the biases described are present among public health professionals, and highlight the need to adjust for them when eliciting their preferences for surveillance systems.