DESIGN AND RESEARCH OF A MULTI-VIEW GRAPH DEEP LEARNING 3D MODEL RETRIEVAL SYSTEM BASED ON FUSION VISION-TRANSFORMER

被引:0
|
作者
Liang, Rong [1 ]
Li, Fangping [1 ]
机构
[1] Taiyuan Univ, Dept Art & Design, 7 Fendong St,Tanghuai Ind Pk, Taiyuan 030000, Peoples R China
关键词
Vision-Transformer; Multi-perspective graph convolutional neural network; 3D; Perspective image; Image entropy;
D O I
10.24507/ijicic.20.06.1775
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The development of computer vision has made three-dimensional models play a crucial role in the field of image processing. However, compared to 2D models, 3D models have more features, making it difficult to extract features and mine correlation information between features. Based on this, this study is based on a multi-perspective graph convolutional neural network, which uses an image entropy weight pooling layer to improve the original view pooling layer. It assigns a weight based on image entropy to each perspective image, and then performs view pooling operations. The Vision-Transformer module is embedded into a multi-perspective graph convolutional neural network to mine information associations between multi-view graphs. The results show that the multi- perspective graph convolutional neural network model fused with Vision-Transformer is more concentrated in classifying features of the same category in the view graph, and there is a significant distance difference between different features. The multi-perspective graph convolutional neural network model fused with Vision-Transformer achieves accuracy of 89.0%, 92.0%, 94.0%, and mean average precision values of 80.0%, 85.0%, and 88.0% when the number of view images is 6, 10, and 14. This study improves the retrieval accuracy of 3D models and has certain reference value in the field of computer vision.
引用
收藏
页码:1775 / 1788
页数:14
相关论文
共 50 条
  • [1] Hierarchical Graph Structure Learning for Multi-View 3D Model Retrieval
    Su, Yuting
    Li, Wenhui
    Liu, Anan
    Nie, Weizhi
    PROCEEDINGS OF THE TWENTY-SEVENTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2018, : 913 - 919
  • [2] SKETCH-BASED 3D SHAPE RETRIEVAL WITH MULTI-VIEW FUSION TRANSFORMER
    Zhu, Cunjuan
    Cui, Dongdong
    Jia, Qi
    Wang, Weimin
    Liu, Yu
    Lew, Michael S.
    2024 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING, ICASSP 2024, 2024, : 3005 - 3009
  • [3] Multi-View Graph Matching for 3D Model Retrieval
    Su, Yu-Ting
    Li, Wen-Hui
    Nie, Wei-Zhi
    Liu, An-An
    ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS, 2020, 16 (03)
  • [4] Multi-view Fusion with Deep Learning for 3D Shape Classification
    Huang, Xiang
    Wang, Mantao
    Zhang, Dejun
    Zhu, Yu
    Zou, Lu
    Sun, Jun
    Han, Fei
    He, Linchao
    2018 INTERNATIONAL CONFERENCE ON AUDIO, LANGUAGE AND IMAGE PROCESSING (ICALIP), 2018, : 189 - 194
  • [5] Deep Semantic Graph Transformer for Multi-View 3D Human Pose Estimation
    Zhang, Lijun
    Zhou, Kangkang
    Lu, Feng
    Zhou, Xiang-Dong
    Shi, Yu
    THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 7, 2024, : 7205 - 7214
  • [6] Group-pair deep feature learning for multi-view 3d model retrieval
    Chen, Xiuxiu
    Liu, Li
    Zhang, Long
    Zhang, Huaxiang
    Meng, Lili
    Liu, Dongmei
    APPLIED INTELLIGENCE, 2022, 52 (02) : 2013 - 2022
  • [7] Group-pair deep feature learning for multi-view 3d model retrieval
    Xiuxiu Chen
    Li Liu
    Long Zhang
    Huaxiang Zhang
    Lili Meng
    Dongmei Liu
    Applied Intelligence, 2022, 52 : 2013 - 2022
  • [8] View-based 3D model retrieval via supervised multi-view feature learning
    An-An Liu
    Yang Shi
    Wei-Zhi Nie
    Yu-Ting Su
    Multimedia Tools and Applications, 2018, 77 : 3229 - 3243
  • [9] View-based 3D model retrieval via supervised multi-view feature learning
    Liu, An-An
    Shi, Yang
    Nie, Wei-Zhi
    Su, Yu-Ting
    MULTIMEDIA TOOLS AND APPLICATIONS, 2018, 77 (03) : 3229 - 3243
  • [10] Multi-view convolutional vision transformer for 3D object recognition
    Li, Jie
    Liu, Zhao
    Li, Li
    Lin, Junqin
    Yao, Jian
    Tu, Jingmin
    JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, 2023, 95