UNSUPERVISED MULTI-TASK LEARNING FOR 3D SUBTOMOGRAM IMAGE ALIGNMENT, CLUSTERING AND SEGMENTATION

被引:0
|
作者
Zhu, Haoyi [1 ]
Wang, Chuting [2 ]
Wang, Yuanxin [3 ]
Fan, Zhaoxin [3 ]
Uddin, Mostofa Rafid [3 ]
Gao, Xin [2 ]
Zhang, Jing [4 ]
Zeng, Xiangrui [3 ]
Xu, Min [3 ]
机构
[1] Shanghai Jiao Tong Univ, Sch Elect Informat & Elect Engn, Shanghai, Peoples R China
[2] King Abdullah Univ Sci & Technol, Struct & Funct Bioinformat Grp, Thuwal, Saudi Arabia
[3] Carnegie Mellon Univ, Sch Comp Sci, Pittsburgh, PA USA
[4] Univ Calif Irvine, Dept Comp Sci, Irvine, CA USA
关键词
unsupervised multi-task learning; subtomogram alignment; subtomogram cluster; subtomogram segmentation; CLASSIFICATION;
D O I
10.1109/ICIP46576.2022.9897919
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
3D subtomogram image alignment, clustering, and segmentation are vital to macromolecular structure recognition in cryo-electron tomography (cryo-ET). However, acquiring ground-truth labels to train a unified deep learning model that can simultaneously deal with these tasks is unaffordable. To this end, we propose an end-to-end unified multi-task learning framework to simultaneously complete the three tasks, where models are trained in an unsupervised manner without using any labels. In particular, we have three parallel branches. In the alignment branch, we adopt a two-stage training scheme, i.e., self-supervised pretraining and constrained unsupervised training using our proposed skip correlation attention layer and constrained loss. Synchronously, in the clustering branch, the learned deep cluster features are utilized to iteratively cluster subtomograms into groups using pseudo-labels from an image-wise Gaussian Mixture Model (GMM). Meanwhile, in the segmentation branch, we use rough pseudo-labels generated from a voxel-wise GMM as supervision signals, and prior knowledge from humans is utilized to jointly learn how to correct these labels as well as predict reliable segmentation results. Benefiting from the end-to-end unified network architecture, our method achieves overall state-of-the-art performance on both simulated and real subtomogram processing benchmarks.
引用
收藏
页码:2751 / 2755
页数:5
相关论文
共 50 条
  • [1] Unsupervised Task Clustering for Multi-task Reinforcement Learning
    Ackermann, Johannes
    Richter, Oliver
    Wattenhofer, Roger
    MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES, 2021, 12975 : 222 - 237
  • [2] Multi-Task Learning for Motion Analysis and Segmentation in 3D Echocardiography
    Ta, Kevinminh
    Ahn, Shawn S.
    Thorn, Stephanie L.
    Stendahl, John C.
    Zhang, Xiaoran
    Langdon, Jonathan
    Staib, Lawrence H.
    Sinusas, Albert J.
    Duncan, James S.
    IEEE TRANSACTIONS ON MEDICAL IMAGING, 2024, 43 (05) : 2010 - 2020
  • [3] 3D Instance Segmentation via Multi-Task Metric Learning
    Lahoud, Jean
    Ghanem, Bernard
    Pollefeys, Marc
    Oswald, Martin R.
    2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, : 9255 - 9265
  • [4] Microcrack Segmentation of 3D CT Images Based on Multi-Task Learning
    Peng, Junjie
    Li, Wenbin
    Liao, Suyu
    Zhu, Yining
    IEEE Access, 2024, 12 : 138192 - 138200
  • [5] Multi-task learning approach for volumetric segmentation and reconstruction in 3D OCT images
    Cahyo, Dheo A. Y.
    Yow, Ai Ping
    Saw, Seang-Mei
    Ang, Marcus
    Girard, Michael
    Schmetterer, Leopold
    Wong, Damon
    BIOMEDICAL OPTICS EXPRESS, 2021, 12 (12) : 7348 - 7360
  • [6] A Multi-task Unsupervised Domain Adaptation Network for Medical Image Segmentation
    Shi, Yuejing
    Zhu, Fan
    Peng, Yan
    Ye, Zhen
    Zhou, Chaozheng
    INTERNATIONAL CONFERENCE ON IMAGE PROCESSING AND INTELLIGENT CONTROL (IPIC 2021), 2021, 11928
  • [7] Gum-Net: Unsupervised Geometric Matching for Fast and Accurate 3D Subtomogram Image Alignment and Averaging
    Zeng, Xiangrui
    Xu, Min
    2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, : 4072 - 4083
  • [8] A Multi-Task Cross-Task Learning Architecture for Ad Hoc Uncertainty Estimation in 3D Cardiac MRI Image Segmentation
    Hasan, S. M. Kamrul
    Linte, Cristian A.
    2021 COMPUTING IN CARDIOLOGY (CINC), 2021,
  • [9] Unsupervised Human Activity Representation Learning with Multi-task Deep Clustering
    Ma, Haojie
    Zhang, Zhijie
    Li, Wenzhong
    Lu, Sanglu
    PROCEEDINGS OF THE ACM ON INTERACTIVE MOBILE WEARABLE AND UBIQUITOUS TECHNOLOGIES-IMWUT, 2021, 5 (01):
  • [10] Action recognition based on 3D motion history image and multi-task learning
    Wang S.
    Dang J.-W.
    Wang Y.-P.
    Jin J.
    Dang, Jian-Wu (dangjw@mail.lzjtu.cn), 1600, Editorial Board of Jilin University (50): : 1495 - 1502