Emotion-Aware Voice Assistants: Design, Implementation, and Preliminary Insights

The increasing prevalence of voice assistants in daily life has transformed human-technology interaction. However, current voice assistants fall short in their ability to comprehend users' emotional states through speech and respond empathetically. This limitation leads to suboptimal user experiences and missed opportunities for personalization. To address this gap, this paper presents a ground-breaking prototype of a future voice assistant equipped with the capability to recognize and respond to users' emotional states. The paper outlines the architecture of a speech-emotion recognition system that utilizes a deep neural network incorporating a sparse attention mechanism, yielding promising results. Furthermore, the paper embarks on an initial exploration of strategies for emotionbased responses, drawing inspiration from human techniques for managing unpleasant emotions. A user study is conducted, exposing participants to emotions such as anger, sadness, and fear, while engaging the voice assistant to elicit happiness. Preliminary findings suggest a tendency toward neutral emotional responses, providing valuable insights into the potential of speech feature analysis as an alternative approach for identifying and responding to emotions.