A deep learning based fusion of RGB camera information and magnetic localization information for endoscopic capsule robots

被引：21
|
作者：
Turan M. [1 ]
Shabbir J. [2 ]
Araujo H. [2 ]
Konukoglu E. [3 ]
Sitti M. [1 ]
机构：
[1] Physical Intelligence Department, Max Planck Institute for Intelligent Systems, Stuttgart
[2] Institute for Systems and Robotics, Universidade de Coimbra, Coimbra
[3] Computer Vision Laboratory, ETH Zurich, Zurich
来源：
International Journal of Intelligent Robotics and Applications | 2017年 / 1卷 / 4期
关键词：
Deep Learning based Sensor Fusion; Endoscopic Capsule Robots; RNN-CNN (RNN:Recurrent Neural Network; CNN: Convolutional Neural Network);
D O I：
10.1007/s41315-017-0039-1
中图分类号：
学科分类号：
摘要：
A reliable, real time localization functionality is crutial for actively controlled capsule endoscopy robots, which are an emerging, minimally invasive diagnostic and therapeutic technology for the gastrointestinal (GI) tract. In this study, we extend the success of deep learning approaches from various research fields to the problem of sensor fusion for endoscopic capsule robots. We propose a multi-sensor fusion based localization approach which combines endoscopic camera information and magnetic sensor based localization information. The results performed on real pig stomach dataset show that our method achieves sub-millimeter precision for both translational and rotational movements. © 2017, The Author(s).
引用
收藏
页码：442 / 450
页数：8
相关论文
共 50 条
  • [1] Magnetic-Visual Sensor Fusion-based Dense 3D Reconstruction and Localization for Endoscopic Capsule Robots
    Turan, Mehmet
    Almalioglu, Yasin
    Ornek, Evin Pinar
    Araujo, Helder
    Yanik, Mehmet Fatih
    Sitti, Metin
    2018 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2018, : 1283 - 1289
  • [2] Endo-VMFuseNet: A Deep Visual-Magnetic Sensor Fusion Approach for Endoscopic Capsule Robots
    Turan, Mehmet
    Almalioglu, Yasin
    Gilbert, Hunter B.
    Sari, Alp Eren
    Soylu, Ufuk
    Sitti, Metin
    2018 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2018, : 5386 - 5392
  • [3] Feature Level Information Fusion Based Deep Learning
    Wang, Kejun
    Hao, Xuesen
    Xing, Xianglei
    PROCEEDINGS OF 2017 CHINESE INTELLIGENT AUTOMATION CONFERENCE, 2018, 458 : 501 - 511
  • [4] Keypoints and Descriptors Based on Cross-Modality Information Fusion for Camera Localization
    MA Shuo
    GAO Yongbin
    TIAN Fangzheng
    LU Junxin
    HUANG Bo
    GU Jia
    ZHOU Yilong
    Wuhan University Journal of Natural Sciences, 2021, 26 (02) : 128 - 136
  • [5] Deep Learning-Based Fault Localization with Contextual Information
    Zhang, Zhuo
    Lei, Yan
    Tan, Qingping
    Mao, Xiaoguang
    Zeng, Ping
    Chang, Xi
    IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2017, E100D (12) : 3027 - 3031
  • [6] Deep Learning-Based Localization with Urban Electromagnetic and Geographic Information
    Wang, Wenyu
    Li, Baozhu
    Huang, Zhen
    Zhu, Lei
    WIRELESS COMMUNICATIONS & MOBILE COMPUTING, 2022, 2022
  • [7] InSAR Phase Unwrapping by Deep Learning Based on Gradient Information Fusion
    Li, Liutong
    Zhang, Hong
    Tang, Yixian
    Wang, Chao
    Gu, Feng
    IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2022, 19
  • [8] An autonomous sewer robots navigation based on stereo camera information
    Ahrary, A
    Tian, L
    Kamata, S
    Ishikawa, M
    ICTAI 2005: 17TH IEEE INTERNATIONAL CONFERENCE ON TOOLS WITH ARTIFICIAL INTELLIGENCE, PROCEEDINGS, 2005, : 628 - 633
  • [9] Real-Time Camera Localization with Deep Learning and Sensor Fusion
    Hu, Tianlun
    Liao, Qi
    IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS (ICC 2021), 2021,
  • [10] Deep-learning-based hyperspectral imaging through a RGB camera
    Gao, Xinyu
    Wang, Tianliang
    Yang, Jing
    Tao, Jinchao
    Qiu, Yanqing
    Meng, Yanlong
    Mao, Bangning
    Zhou, Pengwei
    Li, Yi
    JOURNAL OF ELECTRONIC IMAGING, 2021, 30 (05)
12345