DED-SAM:Adapting Segment Anything Model 2 for Dual Encoder-Decoder Change Detection

被引:1
|
作者
Qiu, Junlong [1 ]
Liu, Wei [1 ]
Zhang, Xin [2 ]
Li, Erzhu [1 ]
Zhang, Lianpeng [1 ]
Li, Xing [1 ]
机构
[1] Jiangsu Normal Univ, Sch Geog Geomat & Planning, Xuzhou 221116, Peoples R China
[2] Chinese Acad Sci, Aerosp Informat Res Inst, Beijing 100094, Peoples R China
基金
国家重点研发计划; 中国国家自然科学基金;
关键词
Feature extraction; Remote sensing; Decoding; Data models; Visualization; Transformers; Object oriented modeling; Accuracy; Semantics; Semantic segmentation; Change detection; remote sensing; segment anything model; vision foundation model; NETWORK;
D O I
10.1109/JSTARS.2024.3490754
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Change detection has become a crucial topic in the field of remote sensing deep learning due to its extensive application in earth observation. However, real remote sensing images often contain multiple land cover classes with significant intraclass variability and interclass similarity, limiting the performance of change detection in complex scenarios. To address this, we leverage the capabilities of vision foundation models by applying the segment anything model (SAM) to remote sensing change detection, and we name this method dual encoder-decoder SAM (DED-SAM). Specifically, we construct a DED framework, utilizing a small-scale change detection model in both branches to generate mixed prompts including image features, mask prompts, and box prompts. The SAM 2 model is used for fine-grained recognition of dual-temporal images, generating accurate, and stable feature boundaries, which are then used as constraints to generate the final change mask. To validate the effectiveness of DED-SAM across various application scenarios, we conduct quantitative experiments on three public datasets: Levir-CD, SYSU-CD, and CDD, testing its detection capabilities under single change categories, multiple change categories, and seasonal pseudochange interference. The results show that the proposed DED-SAM achieved state-of-the-art F1 scores and IoUs on these three datasets: LEVIR-CD (92.00%, 85.11%), SYSU-CD (84.15%, 72.01%), and CDD (97.72%, 95.47%).
引用
收藏
页码:995 / 1006
页数:12
相关论文
共 50 条
  • [1] Encoder-decoder multimodal speaker change detection
    Jung, Jee-weon
    Seo, Soonshin
    Heo, Hee-Soo
    Kim, Geonmin
    Kim, You Jin
    Kwon, Young-ki
    Lee, Minjae
    Lee, Bong-Jin
    INTERSPEECH 2023, 2023, : 5311 - 5315
  • [2] Adapting Segment Anything Model (SAM) for Retinal OCT
    Fazekas, Botond
    Morano, Jose
    Lachinov, Dmitrii
    Aresta, Guilherme
    Bogunovic, Hrvoje
    OPHTHALMIC MEDICAL IMAGE ANALYSIS, OMIA 2023, 2023, 14096 : 92 - 101
  • [3] SCD-SAM: Adapting Segment Anything Model for Semantic Change Detection in Remote Sensing Imagery
    Mei, Liye
    Ye, Zhaoyi
    Xu, Chuan
    Wang, Hongzhu
    Wang, Ying
    Lei, Cheng
    Yang, Wei
    Li, Yansheng
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 : 1 - 13
  • [4] A Dual Attention Encoder-Decoder Text Summarization Model
    Hakami, Nada Ali
    Mahmoud, Hanan Ahmed Hosni
    CMC-COMPUTERS MATERIALS & CONTINUA, 2023, 74 (02): : 3697 - 3710
  • [5] An automated detection system for colonoscopy images using a dual encoder-decoder model
    Hwang, Maxwell
    Wang, Da
    Kong, Xiang-Xing
    Wang, Zhanhuai
    Li, Jun
    Jiang, Wei-Cheng
    Hwang, Kao-Shing
    Ding, Kefeng
    COMPUTERIZED MEDICAL IMAGING AND GRAPHICS, 2020, 84
  • [6] Dual Stream Encoder-Decoder Architecture with Feature Fusion Model for Underwater Object Detection
    Nissar, Mehvish
    Mishra, Amit Kumar
    Subudhi, Badri Narayan
    MATHEMATICS, 2024, 12 (20)
  • [7] Multivariate Segment Expandable Encoder-Decoder Model for Time Series Forecasting
    Li, Yanhong
    Anastasiu, David C.
    IEEE ACCESS, 2024, 12 : 185012 - 185026
  • [8] Adapting Segment Anything Model for Change Detection in VHR Remote Sensing Images
    Ding, Lei
    Zhu, Kun
    Peng, Daifeng
    Tang, Hao
    Yang, Kuiwu
    Bruzzone, Lorenzo
    IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 : 1 - 11
  • [9] UV-SAM: Adapting Segment Anything Model for Urban Village Identification
    Zhang, Xin
    Liu, Yu
    Lin, Yuming
    Liao, Qingmin
    Li, Yong
    THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 20, 2024, : 22520 - 22528
  • [10] Medical SAM adapter: Adapting segment anything model for medical image segmentation
    Wu, Junde
    Wang, Ziyue
    Hong, Mingxuan
    Ji, Wei
    Fu, Huazhu
    Xu, Yanwu
    Xu, Min
    Jin, Yueming
    MEDICAL IMAGE ANALYSIS, 2025, 102