Context-based environmental audio event recognition for scene understanding

被引:1
|
作者
Lu, Tong [1 ]
Wang, Gongyou [1 ]
Su, Feng [1 ]
机构
[1] Nanjing Univ, Dept Comp Sci & Technol, Natl Key Lab Novel Software Technol, Nanjing 210008, Jiangsu, Peoples R China
关键词
Acoustic scene; Audio event; Context modeling; Recognition; 10-CASE; CLASSIFICATION;
D O I
10.1007/s00530-014-0424-7
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Automatic audio content recognition has attracted an increasing attention for developing multimedia systems, for which the most popular approaches combine frame-based features with statistic models or discriminative classifiers. The existing methods are effective for clean single-source event detection but may not perform well for unstructured environmental sounds, which have a broad noise-like flat spectrum and a diverse variety of compositions. We present an automatic acoustic scene understanding framework that detects audio events through two hierarchies, acoustic scene recognition and audio event recognition, in which the former is preceded by following dominant audio sources and in turn helps infer non-dominant audio events within the same scene through modeling their occurrence correlations. On the scene recognition hierarchy, we perform adaptive segmentation and feature extraction for every input acoustic scene stream through Eigen-audiospace and an optimized feature subspace, respectively. After filtering background, scene streams are recognized by modeling the observation density of dominant features using a two-level hidden Markov model. On the audio event recognition hierarchy, scene knowledge is characterized by an audio context model that essentially describes the occurrence correlations of dominant and non-dominant audio events within this scene. Monte Carlo integration and gradient descent techniques are employed to maximize the likelihood and correctly tag each audio event. To the best of our knowledge, this is the first work that models event correlations as scene context for robust audio event detection from complex and noisy environments. Note that according to the recent report, the mean accuracy for the acoustic scene classification task by human listeners is only around 71 % on the data collected in office environments from the DCASE dataset. None of the existing methods performs well on all scene categories and the average accuracy of the best performances of the recent 11 methods is 53.8 %. The proposed method averagely achieves an accuracy of 62.3 % on the same dataset. Additionally, we create a 10-CASE dataset by manually collecting 5,250 audio clips of 10 scene types and 21 event categories. Our experimental results on 10-CASE show that the proposed method averagely achieves the enhanced performance of 78.3 %, and the average accuracy of audio event recognition can be effectively improved by capturing dominant audio sources and reasoning non-dominant events from the dominant ones through acoustic context modeling. In the future work, exploring the interactions between acoustic scene recognition and audio event detection, and incorporating other modalities to improve the accuracy are required to further advance the proposed framework.
引用
下载
收藏
页码:507 / 524
页数:18
相关论文
共 50 条
  • [31] Online Context-based Object Recognition for Mobile Robots
    Ruiz-Sarmiento, J. R.
    Guenther, Martin
    Galindo, Cipriano
    Gonzalez-Jimenez, Javier
    Hertzberg, Joachim
    2017 IEEE INTERNATIONAL CONFERENCE ON AUTONOMOUS ROBOT SYSTEMS AND COMPETITIONS (ICARSC), 2017, : 247 - 252
  • [32] Context-based hand gesture recognition for the operating room
    Jacob, Mithun George
    Wachs, Juan Pablo
    PATTERN RECOGNITION LETTERS, 2014, 36 : 196 - 203
  • [33] Causal neural mechanisms of context-based object recognition
    Wischnewski, Miles
    Peelen, Marius, V
    ELIFE, 2021, 10
  • [34] AUDIO PHRASES FOR AUDIO EVENT RECOGNITION
    Huy Phan
    Hertel, Lars
    Maass, Marco
    Mazur, Radoslaw
    Mertins, Alfred
    2015 23RD EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO), 2015, : 2546 - 2550
  • [35] The use of context-based environmental indicators in corporate reporting
    Haffar, Merriam
    Searcy, Cory
    JOURNAL OF CLEANER PRODUCTION, 2018, 192 : 496 - 513
  • [36] Promoting Environmental Awareness through Context-based Composition
    Philpott, Carolyn
    ORGANISED SOUND, 2018, 23 (01) : 39 - 50
  • [37] Adaptive Audio-Based Context Recognition
    Dargie, Waltenegus
    IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART A-SYSTEMS AND HUMANS, 2009, 39 (04): : 715 - 725
  • [38] Context-based region labeling for event detection in surveillance video
    Javanbakhti, S.
    Zinger, S.
    de With, P. H. N.
    2014 INTERNATIONAL CONFERENCE ON INFORMATION SCIENCE, ELECTRONICS AND ELECTRICAL ENGINEERING (ISEEE), VOLS 1-3, 2014, : 93 - +
  • [39] Adaptive Context Recognition Based on Audio Signal
    Zeng, Zhi
    Li, Xin
    Ma, Xiaohong
    Ji, Qiang
    19TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION, VOLS 1-6, 2008, : 2507 - 2510
  • [40] KITCHEN ACTIVITY RECOGNITION BASED ON SCENE CONTEXT
    Bansal, Shubham
    Khandelwal, Shubham
    Gupta, Shubham
    Goyal, Dushyant
    2013 20TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP 2013), 2013, : 3461 - 3465