Continual Multimodal Knowledge Graph Construction

被引:0
|
作者
Chen, Xiang [1 ,3 ]
Zhang, Jingtian [2 ,3 ]
Wang, Xiaohan [2 ,3 ]
Zhang, Ningyu [2 ,3 ]
Wu, Tongtong [4 ]
Wang, Yuxiang [5 ]
Wang, Yongheng [6 ]
Chen, Huajun [1 ,3 ]
机构
[1] Zhejiang Univ, Coll Comp Sci & Technol, Hangzhou, Peoples R China
[2] Zhejiang Univ, Sch Software Technol, Hangzhou, Peoples R China
[3] Zhejiang Univ, Ant Grp Joint Res Ctr Knowledge Graphs, Hangzhou, Peoples R China
[4] Monash Univ, Melbourne, Vic, Australia
[5] Hangzhou Dianzi Univ, Hangzhou, Peoples R China
[6] Zhejiang Lab, Hangzhou, Peoples R China
基金
中国国家自然科学基金;
关键词
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Current Multimodal Knowledge Graph Construction (MKGC) models struggle with the real-world dynamism of continuously emerging entities and relations, often succumbing to catastrophic forgetting-loss of previously acquired knowledge. This study introduces benchmarks aimed at fostering the development of the continual MKGC domain. We further introduce MSPT framework, designed to surmount the shortcomings of existing MKGC approaches during multimedia data processing. MSPT harmonizes the retention of learned knowledge (stability) and the integration of new data (plasticity), outperforming current continual learning and multimodal methods. Our results confirm MSPT's superior performance in evolving knowledge environments, showcasing its capacity to navigate balance between stability and plasticity.
引用
收藏
页码:6225 / 6233
页数:9
相关论文
共 50 条
  • [1] Continual Learning of Knowledge Graph Embeddings
    Daruna, Angel
    Gupta, Mehul
    Sridharan, Mohan
    Chernova, Sonia
    IEEE ROBOTICS AND AUTOMATION LETTERS, 2021, 6 (02) : 1128 - 1135
  • [2] Multimodal Continual Graph Learning with Neural Architecture Search
    Cai, Jie
    Wang, Xin
    Guan, Chaoyu
    Tang, Yateng
    Xu, Jin
    Zhong, Bin
    Zhu, Wenwu
    PROCEEDINGS OF THE ACM WEB CONFERENCE 2022 (WWW'22), 2022, : 1292 - 1300
  • [3] Multimodal Reasoning with Multimodal Knowledge Graph
    Lee, Junlin
    Wang, Yequan
    Li, Jing
    Zhang, Min
    PROCEEDINGS OF THE 62ND ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 1: LONG PAPERS, 2024, : 10767 - 10782
  • [4] Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI
    Song, Yaoxian
    Sun, Penglei
    Liu, Haoyu
    Li, Zhixu
    Song, Wei
    Xiao, Yanghua
    Zhou, Xiaofang
    IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2024, 36 (11) : 6962 - 6976
  • [5] Multimodal knowledge graph construction for risk identification in water diversion projects
    Wang, Lihu
    Liu, Xuemei
    Liu, Yang
    Li, Hairui
    Liu, Jiaqi
    Yang, Libo
    JOURNAL OF HYDROLOGY, 2024, 635
  • [6] Construction of Multimodal Dialog System via Knowledge Graph in Travel Domain
    Wan, Jing
    Yuan, Minghui
    Dong, Zhenhao
    Hou, Lei
    Xie, Jiawang
    Zhu, Hongyin
    Wen, Qinghua
    WEB AND BIG DATA, PT IV, APWEB-WAIM 2023, 2024, 14334 : 422 - 437
  • [7] Multimodal knowledge graph construction of Chinese traditional operas and sentiment and genre recognition
    Fan, Tao
    Wang, Hao
    Hodel, Tobias
    JOURNAL OF CULTURAL HERITAGE, 2023, 62 : 32 - 44
  • [8] MKGFA: Multimodal Knowledge Graph Construction and Fact-Assisted Reasoning for VQA
    Wang, Longbao
    Zhang, Jinhao
    Zhang, Libing
    Zhang, Shuai
    Xu, Shufang
    Yu, Lin
    Gao, Hongmin
    INTERNATIONAL JOURNAL OF COMPUTATIONAL INTELLIGENCE AND APPLICATIONS, 2024,
  • [9] Survey of Multimodal Knowledge Graph Construction Technology and Its Application in Military Field
    Yao, Yi
    Chen, Zhaoyang
    Du, Xiaoming
    Yao, Tianlei
    Li, Qingshang
    Sun, Mingwei
    Computer Engineering and Applications, 2024, 60 (22) : 18 - 37
  • [10] Query-Driven Knowledge Graph Construction using Question Answering and Multimodal Fusion
    Peng, Yang
    COMPANION OF THE WORLD WIDE WEB CONFERENCE, WWW 2023, 2023, : 1119 - 1126