Deploying Machine Learning Models to Ahead-of-Time Runtime on Edge Using MicroTVM

被引：0

作者：

Liu, Chen ^{[1
]}

Jobst, Matthias ^{[1
]}

Guo, Liyuan ^{[1
]}

Shi, Xinyue ^{[1
]}

Partzsch, Johannes ^{[1
]}

Mayr, Christian ^{[1
]}

机构：

[1] Tech Univ Dresden, Dresden, Germany

来源：

PROCEEDINGS 2023 IEEE/ACM INTERNATIONAL WORKSHOP ON COMPILERS, DEPLOYMENT, AND TOOLING FOR EDGE AI, CODAI 2023 | 2023年

关键词：

TVM; MicroTVM; model deployment; BYOC; UMA;

D O I：

10.1145/3615338.3618125

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

In the past few years, more and more AI applications have been applied to edge devices. However, models trained by data scientists with machine learning frameworks, such as PyTorch or TensorFlow, can not be seamlessly executed on edge. In this paper, we develop an end-to-end code generator parsing a pre-trained model to C source libraries for the backend using MicroTVM, a machine learning compiler framework extension addressing inference on bare metal devices. An analysis shows that specific compute-intensive operators can be easily offloaded to the dedicated accelerator with a Universal Modular Accelerator (UMA) interface, while others are processed in the CPU cores. By using the automatically generated ahead-of-time C runtime, we conduct a hand gesture recognition experiment on an ARM Cortex M4F core.

引用

页码：37 / 40

页数：4

共 50 条

[11] Designing and deploying insurance recommender systems using machine learning
Qazi, Maleeha
Tollas, Kaya
Kanchinadam, Teja
Bockhorst, Joseph
Fung, Glenn
WILEY INTERDISCIPLINARY REVIEWS-DATA MINING AND KNOWLEDGE DISCOVERY, 2020, 10 (04)
[12] DEPLOYR: a technical framework for deploying custom real-time machine learning models into the electronic medical record
Corbin, Conor K.
Maclay, Rob
Acharya, Aakash
Mony, Sreedevi
Punnathanam, Soumya
Thapa, Rahul
Kotecha, Nikesh
Shah, Nigam H.
Chen, Jonathan H.
JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION, 2023, 30 (09) : 1532 - 1542
[13] Strain Prediction of a Bridge Deploying Autoregressive Models with ARIMA and Machine Learning Algorithms
Psathas, Anastasios Panagiotis
Iliadis, Lazaros
Papaleonidas, Antonios
24TH INTERNATIONAL CONFERENCE ON ENGINEERING APPLICATIONS OF NEURAL NETWORKS, EAAAI/EANN 2023, 2023, 1826 : 403 - 419
[14] Optimizing MPI Runtime Parameter Settings by Using Machine Learning
Pellegrini, Simone
Wang, Jie
Fahringer, Thomas
Moritsch, Hans
RECENT ADVANCES IN PARALLEL VIRTUAL MACHINE AND MESSAGE PASSING INTERFACE, PROCEEDINGS, 2009, 5759 : 196 - 206
[15] CBCT Forecasting - A Convolution-LSTM Based Machine Learning Framework for Predicting "ahead-Of-Time" Patient Anatomy, Dose to Be Delivered, and Dose Trend in Head and Neck Radiotherapy
Santhanam, A.
Stiehl, B.
Lauria, M.
Savjani, R.
Gros, S.
Low, D.
MEDICAL PHYSICS, 2022, 49 (06) : E206 - E207
[16] Key Technology Considerations in Developing and Deploying Machine Learning Models in Clinical Radiology Practice
Kulkarni, Viraj
Gawali, Manish
Kharat, Amit
JMIR MEDICAL INFORMATICS, 2021, 9 (09)
[17] Beyond Development: Challenges in Deploying Machine-Learning Models for Structural Engineering Applications
Zaker Esteghamati, Mohsen
Bean, Brennan
Burton, Henry V.
Naser, M.Z.
Journal of Structural Engineering, 2025, 151 (06)
[18] A new algorithm for time series prediction using machine learning models
Jahnavi, Yeturu
Elango, Poongothai
Raja, S. P.
Parra Fuente, Javier
Verdu, Elena
EVOLUTIONARY INTELLIGENCE, 2023, 16 (05) : 1449 - 1460
[19] A new algorithm for time series prediction using machine learning models
Yeturu Jahnavi
Poongothai Elango
S. P. Raja
Javier Parra Fuente
Elena Verdú
Evolutionary Intelligence, 2023, 16 : 1449 - 1460
[20] Adaptive OpenMP Task Scheduling Using Runtime APIs and Machine Learning
Qawasmeh, Ahmad R.
Malik, Abid M.
Chapman, Barbara M.
2015 IEEE 14TH INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND APPLICATIONS (ICMLA), 2015, : 889 - 895

← 1 2 3 4 5 →