Self-Governing Neural Networks for On-Device Short Text Classification

被引:0
|
作者
Ravi, Sujith [1 ]
Kozareva, Zornitsa [2 ]
机构
[1] Google Res, Mountain View, CA 94043 USA
[2] Google, Mountain View, CA 94043 USA
关键词
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Deep neural networks reach state-of-the-art performance for wide range of natural language processing, computer vision and speech applications. Yet, one of the biggest challenges is running these complex networks on devices such as mobile phones or smart watches with tiny memory footprint and low computational capacity. We propose on-device Self-Governing Neural Networks (SGNNs), which learn compact projection vectors with local sensitive hashing. The key advantage of SGNNs over existing work is that they surmount the need for pre-trained word embeddings and complex networks with huge parameters. We conduct extensive evaluation on dialog act classification and show significant improvement over state-of-the-art results. Our findings show that SGNNs are effective at capturing low-dimensional semantic text representations, while maintaining high accuracy.
引用
收藏
页码:804 / 810
页数:7
相关论文
共 50 条
  • [1] Self-Governing Neural Networks for On-Device Short Text Classification
    Ravi, Sujith
    Kozareva, Zornitsa
    [J]. 2018 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2018), 2018, : 887 - 893
  • [2] ProSeqo: Projection Sequence Networks for On-Device Text Classification
    Kozareva, Zornitsa
    Ravi, Sujith
    [J]. 2019 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING AND THE 9TH INTERNATIONAL JOINT CONFERENCE ON NATURAL LANGUAGE PROCESSING (EMNLP-IJCNLP 2019): PROCEEDINGS OF THE CONFERENCE, 2019, : 3894 - 3903
  • [3] Self-supervised short text classification with heterogeneous graph neural networks
    Cao, Meng
    Yuan, Jinliang
    Yu, Hualei
    Zhang, Baoming
    Wang, Chongjun
    [J]. EXPERT SYSTEMS, 2023, 40 (06)
  • [4] On-Device Learning with Binary Neural Networks
    Vorabbi, Lorenzo
    Maltoni, Davide
    Santi, Stefano
    [J]. IMAGE ANALYSIS AND PROCESSING - ICIAP 2023 WORKSHOPS, PT I, 2024, 14365 : 39 - 50
  • [5] Advancements in On-Device Deep Neural Networks
    Saravanan, Kavya
    Kouzani, Abbas Z.
    [J]. INFORMATION, 2023, 14 (08)
  • [6] Hybrid Neural Networks for On-Device Directional Hearing
    Wang, Anran
    Kim, Maruchi
    Zhang, Hao
    Gollakota, Shyamnath
    [J]. THIRTY-SIXTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FOURTH CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE / TWELVETH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2022, : 11421 - 11430
  • [7] On-Device Language Detection and Classification of Extreme Short Text from Calendar Titles Across Languages
    Muni, Rajasekhara Reddy Duvvuru
    Jayakumar, Devanand
    Sivakumar, Tadi Venkata
    Lee, ChangKu
    Hwang, YoungHa
    Kumaraguru, Karthikeyan
    [J]. NATURAL LANGUAGE PROCESSING AND INFORMATION SYSTEMS (NLDB 2022), 2022, 13286 : 47 - 59
  • [8] MEASURES OF DIELECTRIC COEFFICIENTS WITH A SELF-GOVERNING DEVICE USING A PC COMPUTER
    CRAMAN, M
    DAVID, J
    CRAMPAGNE, R
    [J]. REVUE DE PHYSIQUE APPLIQUEE, 1990, 25 (05): : 469 - 474
  • [9] On-Device Learning in Memristor Spiking Neural Networks
    Zyarah, Abdullah M.
    Soures, Nicholas
    Kudithipudi, Dhireesha
    [J]. 2018 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS (ISCAS), 2018,
  • [10] Multi-path Neural Networks for On-device Multi-domain Visual Classification
    Wang, Qifei
    Ke, Junjie
    Greaves, Joshua
    Chu, Grace
    Bender, Gabriel
    Sbaiz, Luciano
    Go, Alec
    Howard, Andrew
    Yang, Ming-Hsuan
    Gilbert, Jeff
    Milanfar, Peyman
    Yang, Feng
    [J]. 2021 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION WACV 2021, 2021, : 3018 - 3027