A Proposed System for Awarding Medals at a Major U.S. Wine Competition

The objective of this research is to devise a reliable and accurate scoring algorithm for awarding medals at a major U.S. wine competition that was described earlier by Hodgson (2008). The new algorithm is comprised of a two-tiered method that considers an award on the basis of a total medal score averaged across four judges per each evaluated wine; and a reliability algorithm that grades the degree of inter- judge reliability according to the following criteria: <70% = Poor or Unacceptable; 70%-79% = Fair; 80%-89% = Good; and 90%-100% = Excellent agreement (Cicchetti, 2001). The method has advantages over a previous scoring method that based its decision upon incorrect estimates of majority opinion; and unacceptable levels of inter-judge agreement.