Unsupervised Bayesian Surprise Detection in Spatial Audio with Convolutional Variational Autoencoder and LSTM Model

Understanding user visual attention (VA) is crucial for Field-of-View (FoV) prediction and resultant bandwidth optimization for 360 degrees video streaming. The influence of spatial audio on VA has been largely overlooked. Traditional methods, using saliency, characterize important stimuli as statistical outliers [4] but fail to capture the Temporal Evolution of Attention (TEA), where initially salient stimuli become routine and less attention-grabbing due to continual exposure [2, 20]. This paper introduces a novel unsupervised deep learning approach using a Convolutional Variational Autoencoder and Long Short-Term Memory (CVAE-LSTM) model to detect Bayesian surprise [2] in spatial audio streams, considering factors such as time, context, and user expectations. Our findings highlight the importance of temporal context in determining the surprisal value of audio events and the selective nature of sensory processing and attention in complex environments.