Text-Guided Knowledge Transfer for Remote Sensing Image-Text Retrieval

被引:0
|
作者
Liu, An-An [1 ,2 ,3 ]
Yang, Bo [1 ]
Li, Wenhui [1 ]
Song, Dan [1 ]
Sun, Zhengya [4 ]
Ren, Tongwei [5 ]
Wei, Zhiqiang [6 ]
机构
[1] Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China
[2] Chinese Acad Sci, Inst Artificial Intelligence, Hefei Comprehens Natl Sci Ctr, Beijing 100045, Peoples R China
[3] Chinese Acad Sci, Key Lab Electromagnet Space Informat, Beijing 100045, Peoples R China
[4] Chinese Acad Sci, Inst Automat, Beijing 100045, Peoples R China
[5] Nanjing Univ, State Key Lab Novel Software Technol, Nanjing 210093, Jiangsu, Peoples R China
[6] Ocean Univ China, Fac Informat Sci & Engn, Qingdao 266005, Shandong, Peoples R China
关键词
CLIP; knowledge transfer; remote sensing image-text retrieval;
D O I
10.1109/LGRS.2024.3374381
中图分类号
P3 [地球物理学]; P59 [地球化学];
学科分类号
0708 ; 070902 ;
摘要
Remote sensing text-image retrieval aims to retrieve valuable information from diverse and complex remote sensing data, attracting significant attention. However, the performance is limited due to the complexity of scenes and their substantial content differences from natural domain images. To address these issues, we propose a simple but effective text-guided knowledge transfer (TGKT) method for remote sensing image-text retrieval. TGKT utilizes CLIP to encode remote sensing data and transfer its rich semantic knowledge from natural to remote sensing domain. The textual information without significant domain differences is employed to bridge the semantic gap between these two domains, thereby enhancing image features. The extensive experimental results on both RSICD and RSITMD datasets demonstrate the effectiveness of our method.
引用
收藏
页码:1 / 5
页数:5
相关论文
共 50 条
  • [1] Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrieval
    Yuan, Yuan
    Zhan, Yang
    Xiong, Zhitong
    [J]. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2023, 61
  • [2] Text-Guided Human Image Manipulation via Image-Text Shared Space
    Xu, Xiaogang
    Chen, Ying-Cong
    Tao, Xin
    Jia, Jiaya
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (10) : 6486 - 6500
  • [3] A Prior Instruction Representation Framework for Remote Sensing Image-text Retrieval
    Pan, Jiancheng
    Ma, Qing
    Bai, Cong
    [J]. PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023, 2023, : 611 - 620
  • [4] MISL: Multi-grained image-text semantic learning for text-guided image inpainting
    Wu, Xingcai
    Zhao, Kejun
    Huang, Qianding
    Wang, Qi
    Yang, Zhenguo
    Hao, Gefei
    [J]. PATTERN RECOGNITION, 2024, 145
  • [5] A FAST AND ACCURATE METHOD FOR REMOTE SENSING IMAGE-TEXT RETRIEVAL BASED ON LARGE MODEL KNOWLEDGE DISTILLATION
    Liao, Yu
    Yang, Rui
    Xie, Tao
    Xing, Hantong
    Quan, Dou
    Wang, Shuang
    Hou, Biao
    [J]. IGARSS 2023 - 2023 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM, 2023, : 5077 - 5080
  • [6] Remote sensing image-text retrieval based on layout semantic joint representation
    Zhang, Ruoyu
    Nie, Jie
    Song, Ning
    Zheng, Chengyu
    Wei, Zhiqiang
    [J]. Beijing Hangkong Hangtian Daxue Xuebao/Journal of Beijing University of Aeronautics and Astronautics, 2024, 50 (02): : 671 - 683
  • [7] Entity Semantic Feature Fusion Network for Remote Sensing Image-Text Retrieval
    Shui, Jianan
    Ding, Shuaipeng
    Li, Mingyong
    Ma, Yan
    [J]. WEB AND BIG DATA, APWEB-WAIM 2024, PT V, 2024, 14965 : 130 - 145
  • [8] Multiscale Salient Alignment Learning for Remote-Sensing Image-Text Retrieval
    Chen, Yaxiong
    Huang, Jinghao
    Li, Xiaoyu
    Xiong, Shengwu
    Lu, Xiaoqiang
    [J]. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 : 1 - 13
  • [9] Remote Sensing Image-Text Retrieval With Implicit-Explicit Relation Reasoning
    Yang, Lingling
    Zhou, Tongqing
    Ma, Wentao
    Du, Mengze
    Liu, Lu
    Li, Feng
    Zhao, Shan
    Wang, Yuwei
    [J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62
  • [10] RELATION-GUIDED NETWORK FOR IMAGE-TEXT RETRIEVAL
    Yang, Yulou
    Shen, Hao
    Yang, Ming
    [J]. 2022 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, ICIP, 2022, : 1856 - 1860