Subjective rating scales in ergonomics: a reply

Comments on the target paper are acknowledged. Whilst there is still some concern that subjective rating scales are scientifically suspect, the general view is that verbal reports, including ratings, constitute objective data, which can be of considerable value in ergonomics research and practice. The main anxiety attached to their use is to ensure acceptable levels of reliability and validity. Quantification in the strict sense can be achieved by some measures but is by no means essential for all scientific and practical purposes. The value of ordinal and qualitative data obtained by subjective judgements should not be underestimated, especially in predicting future performance. Many commonly used constructs such as fatigue, stress, mental workload, usability, etc. are complex and multidimensional, often combining both 'subjective' and 'objective' measures. The validity of individual dimensions and complex constructs lies principally in their relationships with other variables of interest in the context of the specific investigation. The question of design standards based partly or wholly on such scales should therefore be treated with some caution.