Multi-object behaviour recognition based on object detection cascaded image classification in classroom scenes

被引:1
|
作者
Dang, Min [1 ,3 ]
Liu, Gang [1 ,2 ,3 ]
Li, Hao [1 ,3 ]
Xu, Qijie [1 ,3 ]
Wang, Xu [1 ,3 ]
Pan, Rong [1 ,3 ]
机构
[1] Xidian Univ, Sch Life Sci & Technol, 266,Xinglong Sect,Xifeng Rd, Xian 710126, Shaanxi, Peoples R China
[2] Xidian Univ, Guangzhou Inst Technol, 83 Zhiming, Guangzhou 510555, Guangdong, Peoples R China
[3] 266,Xinglong Sect,Xifeng Rd, Xian 710126, Shaanxi, Peoples R China
关键词
Classroom scene; Behaviour recognition; Object detection; Image classification; Vision transformer;
D O I
10.1007/s10489-024-05409-x
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
For multi-object behaviour recognition in classroom scenes, crowded objects have heavy occlusion, invisible keypoints, scale variation, which directly overwhelms the recognition performance. Due to the dense student objects and similar student behaviours, multi-object behaviour recognition brings great challenges. Therefore, we proposed multi-object behaviour recognition based on object detection cascaded image classification. Specifically, object detection extracts student objects, followed by Vision Transformer (ViT) classification of student behaviour. To ensure the accuracy of behaviour recognition, it is first necessary to improve the detection performance of object detection. This paper proposes the Shallow Auxiliary Module for object detection to assist the backbone network in extracting hybrid multi-scale feature information. The multi-scale and multi-channel feature information is fused to alleviate object overlap and scale variation. We propose a Scale Assignment Fusion Mechanism that non-heuristically guides objects to learn the optimal feature layer. Furthermore, the Anchor-free Dynamic Label Assignment can suppress the prediction of low-quality bounding boxes, stabling training and improving detection performance. The proposed student object detector achieves the state-of-the-art mAP50\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$<^>{50}$$\end{document} of 88.03 and APl\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_l$$\end{document} of 57.64, outperforming state-of-the-art object detection methods. Our multi-object behaviour recognition method achieves the recognition of four behaviour classes, which is significantly better than the results of other comparison methods.
引用
收藏
页码:4935 / 4951
页数:17
相关论文
共 50 条
  • [31] RULES OF PHYSICAL PLAUSIBILITY AND RECOGNITION MEMORY FOR MULTI-OBJECT SCENES - DEVELOPMENTAL-STUDY
    HOCK, HS
    ROMANSKI, L
    [J]. BULLETIN OF THE PSYCHONOMIC SOCIETY, 1975, 6 (NB4) : 422 - 423
  • [32] A Multi-object Detection Method Based on Connected Vehicles
    Wang, Yunpeng
    Wang, Xixian
    Tian, Daxin
    Duan, Xuting
    Liu, He
    Gong, Yinsheng
    Sheng, Zhengguo
    Leung, Victor C. M.
    [J]. DIVANET'19: PROCEEDINGS OF THE 9TH ACM SYMPOSIUM ON DESIGN AND ANALYSIS OF INTELLIGENT VEHICULAR NETWORKS AND APPLICATIONS, 2019, : 89 - 96
  • [33] A novel vision-based multi-task robotic grasp detection method for multi-object scenes
    Song, Yanan
    Gao, Liang
    Li, Xinyu
    Shen, Weiming
    Peng, Kunkun
    [J]. SCIENCE CHINA-INFORMATION SCIENCES, 2022, 65 (12)
  • [34] HOG Based Multi-object Detection for Urban Navigation
    Chayeb, A.
    Ouadah, N.
    Tobal, Z.
    Lakrouf, M.
    Azouaoui, O.
    [J]. 2014 IEEE 17TH INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC), 2014, : 2962 - 2967
  • [35] Multi-object Detection Based on Binocular Stereo Vision
    He, Zhannan
    Ren, Qiang
    Yang, Tao
    Li, Jing
    Zhang, Yanning
    [J]. INTELLIGENT VISUAL SURVEILLANCE (IVS 2016), 2016, 664 : 114 - 121
  • [36] A novel vision-based multi-task robotic grasp detection method for multi-object scenes
    Yanan Song
    Liang Gao
    Xinyu Li
    Weiming Shen
    Kunkun Peng
    [J]. Science China Information Sciences, 2022, 65
  • [37] A novel vision-based multi-task robotic grasp detection method for multi-object scenes
    Yanan SONG
    Liang GAO
    Xinyu LI
    Weiming SHEN
    Kunkun PENG
    [J]. Science China(Information Sciences), 2022, 65 (12) : 157 - 169
  • [38] Learned Filters for Object Detection in Multi-object Visual Tracking
    Stamatescu, Victor
    Wong, Sebastien
    McDonnell, Mark D.
    Kearney, David
    [J]. AUTOMATIC TARGET RECOGNITION XXVI, 2016, 9844
  • [39] Envelope detection of multi-object shapes
    Alajlan, N
    El Badawy, O
    Kamel, MS
    Freeman, G
    [J]. IMAGE ANALYSIS AND RECOGNITION, 2005, 3656 : 399 - 406
  • [40] Adaptive superpixel-based multi-object pedestrian recognition
    Tianhe Yu
    Chengdong Wang
    Xiao Liu
    Ming Zhu
    [J]. Machine Vision and Applications, 2021, 32