Can People Self-Report Security Accurately? Agreement Between Self-Report and Behavioral Measures

It is common for researchers to use self-report measures (e.g. surveys) to measure people's security behaviors. In the computer security community, we don't know what behaviors people understand well enough to self-report accurately, or how well those self-reports correlate with what people actually do. In a six week field study, we collected both behavior data and survey responses from 122 subjects. We found that a relatively small number of behaviors - mostly related to tasks that require users to take a specific, regular action - have non-zero correlations. Since security is almost never a user's primary task for everyday computer users, several important security behaviors that we directly measured were not self-reported accurately. These results suggest that security research based on self-report is only reliable for certain behaviors. Additionally, a number of important security behaviors are not sufficiently salient to users that they can self-report accurately.