Multimodal Recipe Recommendation with Heterogeneous Graph Neural Networks

被引:0
|
作者
Ouyang, Ruiqi [1 ]
Huang, Haodong [2 ]
Ou, Weihua [1 ,2 ]
Liu, Qilong [1 ]
机构
[1] Guizhou Normal Univ, Sch Math Sci, Guiyang 550025, Peoples R China
[2] Guizhou Normal Univ, Sch Bigdata & Comp Sci, Guiyang 550025, Peoples R China
关键词
recipe recommendation; heterogeneous graph neural networks; GraphSAGE; multimodal;
D O I
10.3390/electronics13163283
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Recipe recommendation is the process of recommending suitable recipes to users based on factors such as user preferences and dietary needs. Recipes typically involve multiple modalities, with text and images being common, while most typical recipe recommendation methods recommend recipes to users based on text. Obviously, the expressiveness of a single modal is often not enough, and the semantic information of images is more abundant. Moreover, it is difficult to grasp the feature fusion granularity of different kinds of modal information and the relationship between recipes and users. To solve the above problem, this paper proposes a Multimodal Heterogeneous Graph Neural Network Recipe Recommendation (MHGRR) architecture, which aims to fully fuse the various kinds of modal information of recipes and handle the relationship between users and recipes. We use embedding and shallow Convolutional Neural Networks (CNNs) to extract original text and image information for unifying feature fusion granularity, and use Heterogeneous Graph Neural Networks based on GraphSAGE to capture the complex relationship between users and recipes. To verify the effectiveness of our proposed model, we perform some comparative experiments on a real dataset; the experiments show that our method outperforms most popular recipe recommendation methods. Through an ablation experiment, we found that adding image information to recipe recommendation is more effective, and we additionally found that as the output dimensions of GraphSAGE increased, the performance of the model varied little.
引用
收藏
页数:12
相关论文
共 50 条
  • [1] THAN: Multimodal Transportation Recommendation With Heterogeneous Graph Attention Networks
    Xu, Aikun
    Zhong, Ping
    Kang, Yilin
    Duan, Jiongqiang
    Wang, Anning
    Lu, Mingming
    Shi, Chuan
    [J]. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2023, 24 (02) : 1533 - 1543
  • [2] Poincare Heterogeneous Graph Neural Networks for Sequential Recommendation
    Guo, Naicheng
    Liu, Xiaolei
    Li, Shaoshuai
    Ma, Qiongxu
    Gao, Kaixin
    Han, Bing
    Zheng, Lin
    Guo, Sheng
    Guo, Xiaobo
    [J]. ACM TRANSACTIONS ON INFORMATION SYSTEMS, 2023, 41 (03)
  • [3] Preference-aware Heterogeneous Graph Neural Networks for Recommendation
    Fu, Yao
    Wan, Junhong
    Zhao, Hong
    Jiang, Weihao
    Pu, Shiliang
    [J]. 2020 IEEE 32ND INTERNATIONAL CONFERENCE ON TOOLS WITH ARTIFICIAL INTELLIGENCE (ICTAI), 2020, : 41 - 46
  • [4] Session-based Recommendation with Heterogeneous Graph Neural Networks
    Xu, Lei
    Xi, Wu-Dong
    Wang, Chang-Dong
    [J]. 2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2021,
  • [5] Graph Neural Networks for Heterogeneous Trust based Social Recommendation
    Mandal, Supriyo
    Maiti, Abyayananda
    [J]. 2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2021,
  • [6] Multipath-guided heterogeneous graph neural networks for sequential recommendation
    Yin, Fulian
    Xing, Tongtong
    Ji, Meiqi
    Yao, Zebin
    Fu, Ruiling
    Wu, Yuewei
    [J]. COMPUTER SPEECH AND LANGUAGE, 2024, 87
  • [7] Multimodal heterogeneous graph convolutional network for image recommendation
    Weiyi Wei
    Jian Wang
    Mengyu Xu
    Futong Zhang
    [J]. Multimedia Systems, 2023, 29 : 2747 - 2760
  • [8] Multimodal heterogeneous graph convolutional network for image recommendation
    Wei, Weiyi
    Wang, Jian
    Xu, Mengyu
    Zhang, Futong
    [J]. MULTIMEDIA SYSTEMS, 2023, 29 (5) : 2747 - 2760
  • [9] Heterogeneous Global Graph Neural Networks for Personalized Session-based Recommendation
    Pang, Yitong
    Wu, Lingfei
    Shen, Qi
    Zhang, Yiming
    Wei, Zhihua
    Xu, Fangli
    Chang, Ethan
    Long, Bo
    Pei, Jian
    [J]. WSDM'22: PROCEEDINGS OF THE FIFTEENTH ACM INTERNATIONAL CONFERENCE ON WEB SEARCH AND DATA MINING, 2022, : 775 - 783
  • [10] Meta-path guided heterogeneous graph neural networks for news recommendation
    Wang, Feifei
    Lin, Zhongtan
    Wu, Kun
    Han, Shuting
    Sun, Libo
    Lü, Xiaoling
    [J]. Xitong Gongcheng Lilun yu Shijian/System Engineering Theory and Practice, 2024, 44 (05): : 1561 - 1576