On-Device Customization of Tiny Deep Learning Models for Keyword Spotting With Few Examples

被引:2
|
作者
Rusci, Manuele [1 ]
Tuytelaars, Tinne [1 ]
机构
[1] Katholieke Univ Leuven, B-3000 Leuven, Belgium
关键词
All Open Access; Hybrid Gold;
D O I
10.1109/MM.2023.3311826
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Designing a customized keyword spotting (KWS) deep neural network (DNN) for tiny sensors is a time-consuming process, demanding training a new odel on a remote server with a dataset of collected keywords. This article investigates the effectiveness of a DNN-based KWS classifier that can be initialized on-device simply by recording a few examples of the target commands. At runtime, the classifier computes the distance between the DNN output and the prototypes of the recorded keywords. By experimenting with multiple tiny machine learning models on the Google Speech Command dataset, we report an accuracy of up to 80% using only 10 examples of utterances not seen during training. When deployed on a multicore microcontroller with a power envelope of 25 mW, the most accurate ResNet15 model takes 9.7 ms to process a 1-s speech frame, demonstrating the feasibility of on-device KWS customization for tiny devices without requiring any backpropagation-based transfer learning.
引用
收藏
页码:50 / 57
页数:8
相关论文
共 50 条
  • [41] Tiny, Always-on, and Fragile: Bias Propagation through Design Choices in On-device Machine Learning Workflows
    Hutiri, Wiebke
    Ding, Aaron Yi
    Kawsar, Fahim
    Mathur, Akhil
    ACM TRANSACTIONS ON SOFTWARE ENGINEERING AND METHODOLOGY, 2023, 32 (06)
  • [42] Federated Learning with Heterogeneous Models for On-device Malware Detection in IoT Networks
    Shukla, Sanket
    Rafatirad, Setareh
    Homayoun, Houman
    Dinakarrao, Sai Manoj Pudukottai
    2023 DESIGN, AUTOMATION & TEST IN EUROPE CONFERENCE & EXHIBITION, DATE, 2023,
  • [43] Model Extraction Attack against On-device Deep Learning with Power Side Channel
    Liu, Jialin
    Wang, Han
    2024 25TH INTERNATIONAL SYMPOSIUM ON QUALITY ELECTRONIC DESIGN, ISQED 2024, 2024,
  • [44] ARTIFACT: On-Device Deep Learning for IoT-based Wireless Sensing Applications
    Lenka, Manoj
    Chakraborty, Ayon
    2024 IEEE INTERNATIONAL CONFERENCE ON PERVASIVE COMPUTING AND COMMUNICATIONS WORKSHOPS AND OTHER AFFILIATED EVENTS, PERCOM WORKSHOPS, 2024, : 3 - 4
  • [45] FlexDNN: Input-Adaptive On-Device Deep Learning for Efficient Mobile Vision
    Fang, Biyi
    Zeng, Xiao
    Zhang, Faen
    Xu, Hui
    Zhang, Mi
    2020 IEEE/ACM SYMPOSIUM ON EDGE COMPUTING (SEC 2020), 2020, : 84 - 95
  • [46] HARNet: Towards On-Device Incremental Learning using Deep Ensembles on Constrained Devices
    Sundaramoorthy, Prahalathan
    Gudur, Gautham Krishna
    Moorthy, Manav Rajiv
    Bhandari, R. Nidhi
    Vijayaraghavan, Vineeth
    PROCEEDINGS OF THE 2018 INTERNATIONAL WORKSHOP ON EMBEDDED AND MOBILE DEEP LEARNING (EMDL '18), 2018, : 31 - 36
  • [47] On-Device Execution of Deep Learning Models on HoloLens2 for Real-Time Augmented Reality Medical Applications
    Zaccardi, Silvia
    Frantz, Taylor
    Beckwee, David
    Swinnen, Eva
    Jansen, Bart
    SENSORS, 2023, 23 (21)
  • [48] Evolver: A Deep Learning Processor With On-Device Quantization-Voltage-Frequency Tuning
    Tu, Fengbin
    Wu, Weiwei
    Wang, Yang
    Chen, Hongjiang
    Xiong, Feng
    Shi, Man
    Li, Ning
    Deng, Jinyi
    Chen, Tianbao
    Liu, Leibo
    Wei, Shaojun
    Xie, Yuan
    Yin, Shouyi
    IEEE JOURNAL OF SOLID-STATE CIRCUITS, 2021, 56 (02) : 658 - 673
  • [49] On-device facial verification using NUF-Net model of deep learning
    Termritthikun, Chakkrit
    Jamtsho, Yeshi
    Muneesawang, Paisarn
    ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2019, 85 : 579 - 589
  • [50] ON THE EFFICIENCY OF INTEGRATING SELF-SUPERVISED LEARNING AND META-LEARNING FOR USER-DEFINED FEW-SHOT KEYWORD SPOTTING
    Kao, Wei-Tsung
    Wu, Yuan-Kuei
    Chen, Chia-Ping
    Chen, Zhi-Sheng
    Tsai, Yu-Pao
    Lee, Hung-Yi
    2022 IEEE SPOKEN LANGUAGE TECHNOLOGY WORKSHOP, SLT, 2022, : 414 - 421