Character-Level Quantum Mechanical Approach for a Neural Language Model

被引：0

作者：

Wang, Zhihao ^{[1
]}

Ren, Min ^{[2
]}

Tian, Xiaoyan ^{[3
]}

Liang, Xia ^{[1
]}

机构：

[1] Shandong Univ Finance & Econ, Sch Management Sci & Engn, Jinan 250014, Peoples R China

[2] Shandong Univ Finance & Econ, Sch Math & Quantitat Econ, Jinan 250014, Peoples R China

[3] Shandong Police Coll, Jinan 250014, Peoples R China

来源：

INTERNATIONAL JOURNAL OF COMPUTATIONAL INTELLIGENCE SYSTEMS | 2019年 / 12卷 / 02期

基金：

中国国家自然科学基金;

关键词：

Character-level; Quantum theory; Network-in-network; Language model; SEMANTIC ANALYSIS; REPRESENTATIONS;

D O I：

10.2991/ijcis.d.191114.001

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

This article proposes a character-level neural language model (NLM) that is based on quantum theory. The input of the model is the character-level coding represented by the quantum semantic space model. Our model integrates a convolutional neural network (CNN) that is based on network-in-network (NIN). We assessed the effectiveness of our model through extensive experiments based on the English-language Penn Treebank dataset. The experiments results confirm that the quantum semantic inputs work well for the language models. For example, the PPL of our model is 10%-30% less than the states of the arts, while it keeps the relatively smaller number of parameters (i.e., 6 m). (C) 2019 The Authors. Published by Atlantis Press SARL.

引用

页码：1613 / 1621

页数：9

共 50 条

[31] Application of Character-Level Language Models in the Domain of Polish Statutory Law
Smywinski-Pohl, Aleksander
Wrobel, Krzysztof
Lasocki, Karol
Jungiewicz, Michal
LEGAL KNOWLEDGE AND INFORMATION SYSTEMS (JURIX 2019), 2019, 322 : 217 - 222
[32] MALWARE CLASSIFICATION WITH LSTM AND GRU LANGUAGE MODELS AND A CHARACTER-LEVEL CNN
Athiwaratkun, Ben
Stokes, Jack W.
2017 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2017, : 2482 - 2486
[33] Character-Level Neural Translation for Multilingual Media Monitoring in the SUMMA Project
Barzdins, Guntis
Renals, Steve
Gosko, Didzis
LREC 2016 - TENTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION, 2016, : 1789 - 1793
[34] Web Application Firewall using Character-level Convolutional Neural Network
Ito, Michiaki
Iyatomi, Hitoshi
2018 IEEE 14TH INTERNATIONAL COLLOQUIUM ON SIGNAL PROCESSING & ITS APPLICATIONS (CSPA 2018), 2018, : 103 - 106
[35] SanskritWord Segmentation Using Character-level Recurrent and Convolutional Neural Networks
Helwig, Oliver
Nehrdich, Sebastian
2018 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2018), 2018, : 2754 - 2763
[36] A Character-Level Decoder without Explicit Segmentation for Neural Machine Translation
Chung, Junyoung
Cho, Kyunghyun
Bengio, Yoshua
PROCEEDINGS OF THE 54TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 1, 2016, : 1693 - 1703
[37] Character-Level Convolutional Neural Network for Paraphrase Detection and Other Experiments
Maraev, Vladislav
Saedi, Chakaveh
Rodrigues, Joao
Branco, Antonio
Silva, Joao
ARTIFICIAL INTELLIGENCE AND NATURAL LANGUAGE, 2018, 789 : 293 - 304
[38] Tabula Nearly Rasa: Probing the Linguistic Knowledge of Character-level Neural Language Models Trained on Unsegmented Text
Hahn, Michael
Baroni, Marco
TRANSACTIONS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, 2019, 7 : 467 - 484
[39] Improving Bug Localization with Character-level Convolutional Neural Network and Recurrent Neural Network
Xiao, Yan
Keung, Jacky
2018 25TH ASIA-PACIFIC SOFTWARE ENGINEERING CONFERENCE (APSEC 2018), 2018, : 703 - 704
[40] A Novel Joint Character Categorization and Localization Approach for Character-Level Scene Text Recognition
Qi, Xianbiao
Chen, Yihao
Xiao, Rong
Li, Chun-Guang
Zou, Qin
Cui, Shuguang
2019 INTERNATIONAL CONFERENCE ON DOCUMENT ANALYSIS AND RECOGNITION WORKSHOPS (ICDARW), VOL 5, 2019, : 83 - 90

← 1 2 3 4 5 →