Text-Guided Knowledge Transfer for Remote Sensing Image-Text Retrieval

被引:0
|
作者
Liu, An-An [1 ,2 ,3 ]
Yang, Bo [1 ]
Li, Wenhui [1 ]
Song, Dan [1 ]
Sun, Zhengya [4 ]
Ren, Tongwei [5 ]
Wei, Zhiqiang [6 ]
机构
[1] Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China
[2] Chinese Acad Sci, Inst Artificial Intelligence, Hefei Comprehens Natl Sci Ctr, Beijing 100045, Peoples R China
[3] Chinese Acad Sci, Key Lab Electromagnet Space Informat, Beijing 100045, Peoples R China
[4] Chinese Acad Sci, Inst Automat, Beijing 100045, Peoples R China
[5] Nanjing Univ, State Key Lab Novel Software Technol, Nanjing 210093, Jiangsu, Peoples R China
[6] Ocean Univ China, Fac Informat Sci & Engn, Qingdao 266005, Shandong, Peoples R China
关键词
CLIP; knowledge transfer; remote sensing image-text retrieval;
D O I
10.1109/LGRS.2024.3374381
中图分类号
P3 [地球物理学]; P59 [地球化学];
学科分类号
0708 ; 070902 ;
摘要
Remote sensing text-image retrieval aims to retrieve valuable information from diverse and complex remote sensing data, attracting significant attention. However, the performance is limited due to the complexity of scenes and their substantial content differences from natural domain images. To address these issues, we propose a simple but effective text-guided knowledge transfer (TGKT) method for remote sensing image-text retrieval. TGKT utilizes CLIP to encode remote sensing data and transfer its rich semantic knowledge from natural to remote sensing domain. The textual information without significant domain differences is employed to bridge the semantic gap between these two domains, thereby enhancing image features. The extensive experimental results on both RSICD and RSITMD datasets demonstrate the effectiveness of our method.
引用
下载
收藏
页码:1 / 5
页数:5
相关论文
共 50 条
  • [1] Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrieval
    Yuan, Yuan
    Zhan, Yang
    Xiong, Zhitong
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2023, 61
  • [2] Text-Guided Human Image Manipulation via Image-Text Shared Space
    Xu, Xiaogang
    Chen, Ying-Cong
    Tao, Xin
    Jia, Jiaya
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (10) : 6486 - 6500
  • [3] Visual Global-Salient-Guided Network for Remote Sensing Image-Text Retrieval
    He, Yangpeng
    Xu, Xin
    Chen, Hongjia
    Li, Jinwen
    Pu, Fangling
    IEEE Transactions on Geoscience and Remote Sensing, 2024, 62
  • [4] A Prior Instruction Representation Framework for Remote Sensing Image-text Retrieval
    Pan, Jiancheng
    Ma, Qing
    Bai, Cong
    PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023, 2023, : 611 - 620
  • [5] MISL: Multi-grained image-text semantic learning for text-guided image inpainting
    Wu, Xingcai
    Zhao, Kejun
    Huang, Qianding
    Wang, Qi
    Yang, Zhenguo
    Hao, Gefei
    PATTERN RECOGNITION, 2024, 145
  • [6] Remote sensing image-text retrieval based on layout semantic joint representation
    Zhang R.
    Nie J.
    Song N.
    Zheng C.
    Wei Z.
    Beijing Hangkong Hangtian Daxue Xuebao/Journal of Beijing University of Aeronautics and Astronautics, 2024, 50 (02): : 671 - 683
  • [7] A FAST AND ACCURATE METHOD FOR REMOTE SENSING IMAGE-TEXT RETRIEVAL BASED ON LARGE MODEL KNOWLEDGE DISTILLATION
    Liao, Yu
    Yang, Rui
    Xie, Tao
    Xing, Hantong
    Quan, Dou
    Wang, Shuang
    Hou, Biao
    IGARSS 2023 - 2023 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM, 2023, : 5077 - 5080
  • [8] Transcending Fusion: A Multiscale Alignment Method for Remote Sensing Image-Text Retrieval
    Yang, Rui
    Wang, Shuang
    Han, Yingping
    Li, Yuanheng
    Zhao, Dong
    Quan, Dou
    Guo, Yanhe
    Jiao, Licheng
    Yang, Zhi
    IEEE Transactions on Geoscience and Remote Sensing, 2024, 62
  • [9] Multiscale Salient Alignment Learning for Remote-Sensing Image-Text Retrieval
    Chen, Yaxiong
    Huang, Jinghao
    Li, Xiaoyu
    Xiong, Shengwu
    Lu, Xiaoqiang
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 : 1 - 13
  • [10] Entity Semantic Feature Fusion Network for Remote Sensing Image-Text Retrieval
    Shui, Jianan
    Ding, Shuaipeng
    Li, Mingyong
    Ma, Yan
    WEB AND BIG DATA, APWEB-WAIM 2024, PT V, 2024, 14965 : 130 - 145