Exploring Semi-Supervised Methods for Labeling Support in Multimodal Datasets

被引:4
|
作者
Diete, Alexander [1 ]
Sztyler, Timo [1 ]
Stuckenschmidt, Heiner [1 ]
机构
[1] Univ Mannheim, Data & Web Sci Grp, D-68131 Mannheim, Germany
关键词
machine learning; activity recognition; multimodal labeling;
D O I
10.3390/s18082639
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
Working with multimodal datasets is a challenging task as it requires annotations which often are time consuming and difficult to acquire. This includes in particular video recordings which often need to be watched as a whole before they can be labeled. Additionally, other modalities like acceleration data are often recorded alongside a video. For that purpose, we created an annotation tool that enables to annotate datasets of video and inertial sensor data. In contrast to most existing approaches, we focus on semi-supervised labeling support to infer labels for the whole dataset. This means, after labeling a small set of instances our system is able to provide labeling recommendations. We aim to rely on the acceleration data of a wrist-worn sensor to support the labeling of a video recording. For that purpose, we apply template matching to identify time intervals of certain activities. We test our approach on three datasets, one containing warehouse picking activities, one consisting of activities of daily living and one about meal preparations. Our results show that the presented method is able to give hints to annotators about possible label candidates.
引用
收藏
页数:18
相关论文
共 50 条
  • [1] Semi-supervised labeling: a proposed methodology for labeling the twitter datasets
    Jan, Tabassum Gull
    Khurana, Surinder Singh
    Kumar, Munish
    [J]. MULTIMEDIA TOOLS AND APPLICATIONS, 2022, 81 (06) : 7669 - 7683
  • [2] Semi-supervised labeling: a proposed methodology for labeling the twitter datasets
    Tabassum Gull Jan
    Surinder Singh Khurana
    Munish Kumar
    [J]. Multimedia Tools and Applications, 2022, 81 : 7669 - 7683
  • [3] Semi-supervised Learning with Multimodal Perturbation
    Su, Lei
    Liao, Hongzhi
    Yu, Zhengtao
    Tang, Jiahua
    [J]. ADVANCES IN NEURAL NETWORKS - ISNN 2009, PT 1, PROCEEDINGS, 2009, 5551 : 651 - +
  • [4] Semi-supervised Mesh Segmentation and Labeling
    Lv, Jiajun
    Chen, Xinlei
    Huang, Jin
    Bao, Hujun
    [J]. COMPUTER GRAPHICS FORUM, 2012, 31 (07) : 2241 - 2248
  • [5] Semi-supervised methods for exploring the acoustics of simple productive feedback
    Neiberg, Daniel
    Salvi, Giampiero
    Gustafson, Joakim
    [J]. SPEECH COMMUNICATION, 2013, 55 (03) : 451 - 469
  • [6] Semi-supervised Classification for Remote Sensing Datasets
    Hernandez-Sequeira, Itza
    Fernandez-Beltran, Ruben
    Xu, Yonghao
    Ghamisi, Pedram
    Pla, Filiberto
    [J]. IMAGE ANALYSIS AND PROCESSING, ICIAP 2023, PT I, 2023, 14233 : 463 - 474
  • [7] Semi-Supervised Multimodal Emotion Recognition with Class-Balanced Pseudo-Labeling
    Chen, Haifeng
    Guo, Chujia
    Li, Yan
    Zhang, Peng
    Jiang, Dongmei
    [J]. PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023, 2023, : 9556 - 9560
  • [8] Semi-supervised clustering methods
    Bair, Eric
    [J]. WILEY INTERDISCIPLINARY REVIEWS-COMPUTATIONAL STATISTICS, 2013, 5 (05): : 349 - 361
  • [9] Semi-supervised Multitask Learning for Sequence Labeling
    Rei, Marek
    [J]. PROCEEDINGS OF THE 55TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2017), VOL 1, 2017, : 2121 - 2130
  • [10] Multimodal semi-supervised learning for image classification
    Guillaumin, Matthieu
    Verbeek, Jakob
    Schmid, Cordelia
    [J]. 2010 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2010, : 902 - 909