Impact of Web Survey Invitation Design on Survey Participation, Respondents, and Survey Responses

This article investigates how webpages shown to respondents before they enter a self-administered survey impact whether and how they respond. Using the SurveyMonkey end page, a page that is displayed nearly 3 million times daily to survey respondents who have completed a user-created survey, this article examines whether the type of image and text that are displayed on the end page has a large impact on survey participation, whether different images/texts appeal to different types of respondents, and whether they render different substance responses. The topic of the survey is about the U.K. 2015 General Election. The findings show that when a concrete image/text combination is displayed, the click rate is lower than more general and abstract image/text combinations. The completion rates across different webpages are similar. We also find that the sample compositions and selected survey responses differ by different end pages. As a last step, we conducted poststratification in each condition separately to produce estimates that match known population proportions for selected demographic characteristics. After weighting adjustment, the differences in substantive responses disappeared or reduced, suggesting that different people drawn into the survey through different webpages contribute to the different substantive responses.