Comparing Web and Mail Protocols for Administering Hospital Consumer Assessment of Healthcare Providers and Systems Surveys

Objective: The objective of this study was to compare results of using web-based and mail (postal) Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) data collection protocols. Research Design: Patients who had been hospitalized in a New England Hospital were surveyed about their hospital experience. Patients who provided email addresses were randomized to 1 of 3 data collection protocols: web-alone, web with postal mail follow-up, and postal mail only. Those who did not provide email addresses were surveyed using postal mail only. Analyses compared response rates, respondent characteristics, and patient-reported experiences. Subjects: For an 8-week period, patients were discharged from the study hospital to home. Measures: Measures included response rates, characteristics of respondents, 6 composite measures of their patient experiences, and 2 ratings of the hospital. Results: Response rates were significantly lower for the web-only protocol than the mail or combined protocols, and those who had not provided email addresses had lower response rates. Those over 65 were more likely than others to respond to all protocols, especially for the mail-only protocols. Respondents without email addresses were older, less educated, and reported worse health than those who provided email addresses. After adjusting for respondent differences, those in the combined protocol differed significantly from the mail (postal) only respondents on 2 measures of patient experience; those in the web-only protocol differed on one. Those not providing an email address differed from those who did on one measure. Conclusion: If web-based protocols are used for HCAHPS surveys, adjustments for a mode of data collection are needed to make results comparable.