3MASSIV Multilingual, Multimodal and Multi-Aspect dataset of Social Media Short Videos

被引:1
|
作者
Gupta, Vikram [1 ]
Mittal, Trisha [2 ]
Mathur, Puneet [2 ]
Mishra, Vaibhav [1 ]
Maheshwari, Mayank [1 ]
Bera, Aniket [2 ]
Mukherjee, Debdoot [1 ]
Manocha, Dinesh [2 ]
机构
[1] ShareChat, Bangalore, Karnataka, India
[2] Univ Maryland, College Pk, MD 20742 USA
关键词
VISUALLY GROUNDED SPEECH; EMOTION RECOGNITION;
D O I
10.1109/CVPR52688.2022.02039
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We present 3MASSIV, a multilingual, multimodal and multi-aspect, expertly-annotated dataset of diverse short videos extracted from short-video social media platform Maj. 3MASSIV comprises of 50k short videos (20 seconds average duration) and 100K unlabeled videos in 11 different languages and captures popular short video trends like pranks, fails, romance, comedy expressed via unique audio-visual formats like self-shot videos, reaction videos, lip-synching, self-sung songs, etc. 3MASSIV presents an opportunity for multimodal and multilingual semantic understanding on these unique videos by annotating them for concepts, affective states, media types, and audio language. We present a thorough analysis of 3MASSIV and highlight the variety and unique aspects of our dataset compared to other contemporary popular datasets with strong baselines. We also show how the social media content in 3MASSIV is dynamic and temporal in nature, which can be used for semantic understanding tasks and cross-lingual analysis.
引用
收藏
页码:21032 / 21043
页数:12
相关论文
共 6 条
  • [1] Flood Detection in Social Media Using Multimodal Fusion on Multilingual Dataset
    Jony, Rabiul Islam
    Woodley, Alan
    Perrin, Dimitri
    [J]. 2021 INTERNATIONAL CONFERENCE ON DIGITAL IMAGE COMPUTING: TECHNIQUES AND APPLICATIONS (DICTA 2021), 2021, : 566 - 573
  • [2] Multi-aspect Entity-Centric Analysis of Big Social Media Archives
    Fafalios, Pavlos
    Iosifidis, Vasileios
    Stefanidis, Kostas
    Ntoutsi, Eirini
    [J]. RESEARCH AND ADVANCED TECHNOLOGY FOR DIGITAL LIBRARIES (TPDL 2017), 2017, 10450 : 261 - 273
  • [3] Multi-Aspect Transfer Learning for Detecting Low Resource Mental Disorders on Social Media
    Uban, Ana Sabina
    Chulvi, Berta
    Rosso, Paolo
    [J]. LREC 2022: THIRTEEN INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION, 2022, : 3202 - 3219
  • [4] Explainable depression detection with multi-aspect features using a hybrid deep learning model on social media
    Hamad Zogan
    Imran Razzak
    Xianzhi Wang
    Shoaib Jameel
    Guandong Xu
    [J]. World Wide Web, 2022, 25 : 281 - 304
  • [5] Explainable depression detection with multi-aspect features using a hybrid deep learning model on social media
    Zogan, Hamad
    Razzak, Imran
    Wang, Xianzhi
    Jameel, Shoaib
    Xu, Guandong
    [J]. WORLD WIDE WEB-INTERNET AND WEB INFORMATION SYSTEMS, 2022, 25 (01): : 281 - 304
  • [6] AMPS: Predicting popularity of short-form videos using multi-modal attention mechanisms in social media marketing environments
    Cho, Minhwa
    Jeong, Dahye
    Park, Eunil
    [J]. JOURNAL OF RETAILING AND CONSUMER SERVICES, 2024, 78