Detecting Careless Responses to Self-Reported Questionnaires

Problem Statement: The use of self-report questionnaires may lead to biases such as careless responses that distort the research outcomes. Early detection of careless responses in self-report questionnaires may reduce error, but little guidance exists in the literature regarding techniques for detecting such careless or random responses in self-report questionnaires. Purpose of the Study: The purpose of this study was to examine whether the respondent's goodness-of-fit test score (R-GF) can be used to indicate careless responses in completing self-report questionnaires. It is hypothesized that there is a significant difference of RGF between careless responses and true responses and that RGF of careless responses is higher than RGF for true responses. Method: An experimental research design that made use of a self-reported questionnaire was conducted with 205 respondents divided into two groups. The first group responded truthfully to the questionnaire while the second group responded carelessly to the questionnaire. The validity and reliability of the questionnaire had been tested. One hundred and eighty five respondents were selected as the group of true responses, while another 20 respondents comprised the group of careless responses. T-test of independent sample was used to evaluate the different RGF among true responses and careless responses. Findings: After comparing the mean scores of RGF between careless responses and true responses, a significant difference was found. The frequency distribution of true responses tends to be normally distributed while the existence of careless responses creates a skewed distribution to the right. The R-GF of careless responses is higher than the RGF of true responses. Conclusion and Recommendations: RGF may be used as an indicator of respondent's careless responses in self-report questionnaires in which more accurate data are expected. Social science research that makes use of self-report questionnaire in measuring affective domain may compute RGF to determine whether careless responses exist.