Accelerating Human-in-the-loop Machine Learning: Challenges and Opportunities

被引：54
|
作者：
Xin, Doris [1 ]
Ma, Litian [1 ]
Liu, Jialin [1 ]
Macke, Stephen [1 ]
Song, Shuchen [1 ]
Parameswaran, Aditya [1 ]
机构：
[1] Univ Illinois Urbana Champaign UIUC, Champaign, IL 61820 USA
来源：
PROCEEDINGS OF THE SECOND WORKSHOP ON DATA MANAGEMENT FOR END-TO-END MACHINE LEARNING | 2018年
基金：
美国国家科学基金会;
关键词：
D O I：
10.1145/3209889.3209897
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
Development of machine learning (ML) workflows is a tedious process of iterative experimentation: developers repeatedly make changes to workflows until the desired accuracy is attained. We describe our vision for a "human-in-the-loop" ML system that accelerates this process: by intelligently tracking changes and intermediate results over time, such a system can enable rapid iteration, quick responsive feedback, introspection and debugging, and background execution and automation. We finally describe Helix, our preliminary attempt at such a system that has already led to speedups of upto 10x on typical iterative workflows against competing systems.
引用
收藏
页数：4
相关论文
共 50 条
  • [21] Understanding the Relationship between Interactions and Outcomes in Human-in-the-Loop Machine Learning
    Cui, Yuchen
    Koppol, Pallavi
    Admoni, Henny
    Niekum, Scott
    Simmons, Reid
    Steinfeld, Aaron
    Fitzgerald, Tesca
    PROCEEDINGS OF THE THIRTIETH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, IJCAI 2021, 2021, : 4382 - 4391
  • [22] Using Segmentation to Improve Machine Learning Performance in Human-in-the-Loop Systems
    Carneiro, Davide
    Carvalho, Mariana
    INTELLIGENT SYSTEMS AND APPLICATIONS, VOL 2, 2023, 543 : 413 - 428
  • [23] Challenges in Data Production for AI with Human-in-the-Loop
    Ustalov, Dmitry
    WSDM'22: PROCEEDINGS OF THE FIFTEENTH ACM INTERNATIONAL CONFERENCE ON WEB SEARCH AND DATA MINING, 2022, : 1651 - 1652
  • [24] Human-in-the-Loop Machine Learning Curation of Problems That Bother Parkinson Disease Patients
    Marras, Connie
    Arbatti, Lakshmi
    Amara, Amy
    Anderson, Karen
    Bale, Claire
    Chahine, Lana
    Eberly, Shirley
    Hosamath, Abhishek
    Kinel, Daniel
    Mantri, Sneha
    Mathur, Soania
    Oakes, David
    Purks, Jennifer
    Weintraub, Daniel
    Shoulson, Ira
    ANNALS OF NEUROLOGY, 2022, 92 : S116 - S116
  • [25] Toward Human-in-the-Loop Collaboration Between Software Engineers and Machine Learning Algorithms
    Nascimento, Nathalia
    Alencar, Paulo
    Lucena, Carlos
    Cowan, Donald
    2018 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2018, : 3534 - 3540
  • [26] Increasing Video Accessibility for Visually Impaired Users with Human-in-the-Loop Machine Learning
    Yuksel, Beste F.
    Kim, Soo Jung
    Jin, Seung Jung
    Fazli, Pooyan
    Mathur, Umang
    Bisht, Vaishali
    Yoon, Ilmi
    Siu, Yue-Ting
    Lee, Joshua Junhee
    Miele, Joshua A.
    CHI'20: EXTENDED ABSTRACTS OF THE 2020 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, 2020,
  • [27] Human-in-the-Loop AI Reviewing: Feasibility, Opportunities, and Risks
    Drori, Iddo
    Te'eni, Dov
    JOURNAL OF THE ASSOCIATION FOR INFORMATION SYSTEMS, 2024, 25 (01): : 98 - 109
  • [28] The Human in the Smart Factory Human-in-The-Loop: A Human-centered Approach to Knowledge Augmentation with Machine Learning
    Lück M.
    Hornung T.
    Teklezgi J.
    ZWF Zeitschrift fuer Wirtschaftlichen Fabrikbetrieb, 2024, 119 (06): : 456 - 459
  • [29] Interactive machine learning for health informatics: when do we need the human-in-the-loop?
    Holzinger A.
    Brain Informatics, 2016, 3 (2) : 119 - 131
  • [30] Continual learning classification method with human-in-the-loop
    Liu, Jia
    Li, Dong
    Shan, Wangweiyi
    Liu, Shulin
    METHODSX, 2023, 11
12345