The Effects of Risk and Role on Users' Anticipated Emotions in Safety-Critical Systems

Users of safety-critical systems often need to make risky decisions in real-time. However, current system designs do not sufficiently take users' emotions into account. This lack of consideration may negatively influence a user's decision-making and undermine the effectiveness of such a "human-computer collaboration." In a two-way, 2 (role: operator/system administrator) x 3 (risk level: high/medium/low) factorial study, we investigated the intensity of 44 emotions anticipated by 296 Mechanical Turk users who imagined being the (1) operator or (2) administrator of a drone system identifying (a) enemies on a battlefield, (b) illegal immigrants or (c) whale pods. Results indicated that risk level had a significant main effect on ratings of negative individualistic and negative prosocial emotions. Participants assigned to the high risk scenario anticipated more intense negative individualistic (e.g., nervous) and negative prosocial (e.g., resentful, lonely) emotions and less intense positive (e.g., happy, proud) emotions than participants assigned to the medium and low risk scenarios. We discuss the implications of our findings for the design of safety-critical systems.