Using DeepLabCut for 3D markerless pose estimation across species and behaviors

被引:0
|
作者
Tanmay Nath
Alexander Mathis
An Chi Chen
Amir Patel
Matthias Bethge
Mackenzie Weygandt Mathis
机构
[1] Rowland Institute at Harvard,Department of Molecular & Cellular Biology
[2] Harvard University,Department of Electrical Engineering
[3] Harvard University,Tübingen AI Center & Centre for Integrative Neuroscience
[4] University of Cape Town,undefined
[5] Eberhard Karls Universität Tübingen,undefined
来源
Nature Protocols | 2019年 / 14卷
关键词
D O I
暂无
中图分类号
学科分类号
摘要
Noninvasive behavioral tracking of animals during experiments is critical to many scientific pursuits. Extracting the poses of animals without using markers is often essential to measuring behavioral effects in biomechanics, genetics, ethology, and neuroscience. However, extracting detailed poses without markers in dynamically changing backgrounds has been challenging. We recently introduced an open-source toolbox called DeepLabCut that builds on a state-of-the-art human pose-estimation algorithm to allow a user to train a deep neural network with limited training data to precisely track user-defined features that match human labeling accuracy. Here, we provide an updated toolbox, developed as a Python package, that includes new features such as graphical user interfaces (GUIs), performance improvements, and active-learning-based network refinement. We provide a step-by-step procedure for using DeepLabCut that guides the user in creating a tailored, reusable analysis pipeline with a graphical processing unit (GPU) in 1–12 h (depending on frame size). Additionally, we provide Docker environments and Jupyter Notebooks that can be run on cloud resources such as Google Colaboratory.
引用
收藏
页码:2152 / 2176
页数:24
相关论文
共 50 条
  • [1] Using DeepLabCut for 3D markerless pose estimation across species and behaviors
    Nath, Tanmay
    Mathis, Alexander
    Chen, An Chi
    Patel, Amir
    Bethge, Matthias
    Mathis, Mackenzie Weygandt
    [J]. NATURE PROTOCOLS, 2019, 14 (07) : 2152 - 2176
  • [2] Primate Markerless Pose Estimation and Movement Analysis Using DeepLabCut
    Labuguen, Rollyn
    Bardeloza, Dean Karlo
    Negrete, Salvador Blanco
    Matsumoto, Jumpei
    Inoue, Kenichi
    Shibata, Tomohiro
    [J]. 2019 JOINT 8TH INTERNATIONAL CONFERENCE ON INFORMATICS, ELECTRONICS & VISION (ICIEV) AND 2019 3RD INTERNATIONAL CONFERENCE ON IMAGING, VISION & PATTERN RECOGNITION (ICIVPR) WITH INTERNATIONAL CONFERENCE ON ACTIVITY AND BEHAVIOR COMPUTING (ABC), 2019, : 297 - 300
  • [3] Markerless 3D Pose Estimation System for Mouse Musculoskeletal Model Using DeepLabCut and Multiple RGB-D Cameras
    Tsuruda, Yoshito
    Akita, Shingo
    Yamanaka, Kotomi
    Yamamoto, Masataka
    Sano, Yoshitake
    Furuichi, Teiichi
    Takemura, Hiroshi
    [J]. 2023 IEEE/SICE INTERNATIONAL SYMPOSIUM ON SYSTEM INTEGRATION, SII, 2023,
  • [4] Anipose: A toolkit for robust markerless 3D pose estimation
    Karashchuk, Pierre
    Rupp, Katie L.
    Dickinson, Evyn S.
    Walling-Bell, Sarah
    Sanders, Elischa
    Azim, Eiman
    Brunton, Bingni W.
    Tuthill, John C.
    [J]. CELL REPORTS, 2021, 36 (13):
  • [5] Improved 3D Markerless Mouse Pose Estimation Using Temporal Semi-supervision
    Li, Tianqing
    Severson, Kyle S. S.
    Wang, Fan
    Dunn, Timothy W. W.
    [J]. INTERNATIONAL JOURNAL OF COMPUTER VISION, 2023, 131 (06) : 1389 - 1405
  • [6] The reliability and validity of gait analysis system using 3D markerless pose estimation algorithms
    Liang, Shengyun
    Zhang, Yu
    Diao, Yanan
    Li, Guanglin
    Zhao, Guoru
    [J]. FRONTIERS IN BIOENGINEERING AND BIOTECHNOLOGY, 2022, 10
  • [7] Improved 3D Markerless Mouse Pose Estimation Using Temporal Semi-supervision
    Tianqing Li
    Kyle S. Severson
    Fan Wang
    Timothy W. Dunn
    [J]. International Journal of Computer Vision, 2023, 131 : 1389 - 1405
  • [8] Beyond the Edge: Markerless Pose Estimation of Speech Articulators from Ultrasound and Camera Images Using DeepLabCut
    Wrench, Alan
    Balch-Tomes, Jonathan
    [J]. SENSORS, 2022, 22 (03)
  • [9] A review of 3D human pose estimation algorithms for markerless motion capture
    Desmarais, Yann
    Mottet, Denis
    Slangen, Pierre
    Montesinos, Philippe
    [J]. COMPUTER VISION AND IMAGE UNDERSTANDING, 2021, 212
  • [10] Validating markerless pose estimation with 3D X-ray radiography
    Moore, Dalton D.
    Walker, Jeffrey D.
    MacLean, Jason N.
    Hatsopoulos, Nicholas G.
    [J]. JOURNAL OF EXPERIMENTAL BIOLOGY, 2022, 225 (09):