Structural Properties of Recurrent Neural Networks

被引:1
|
作者
Dobnikar, Andrej [1 ]
Ster, Branko [1 ]
机构
[1] Univ Ljubljana, Fac Comp & Informat Sci, Ljubljana 1000, Slovenia
关键词
Recurrent neural networks; Complex systems; Graph theory; Dynamical systems; ALGORITHM; AUTOMATA;
D O I
10.1007/s11063-009-9096-2
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this article we research the impact of the adaptive learning process of recurrent neural networks (RNN) on the structural properties of the derived graphs. A trained fully connected RNN can be converted to a graph by defining edges between pairs od nodes having significant weights. We measured structural properties of the derived graphs, such as characteristic path lengths, clustering coefficients and degree distributions. The results imply that a trained RNN has significantly larger clustering coefficient than a random network with a comparable connectivity. Besides, the degree distributions show existence of nodes with a large degree or hubs, typical for scale-free networks. We also show analytically and experimentally that this type of degree distribution has increased entropy.
引用
收藏
页码:75 / 88
页数:14
相关论文
共 50 条
  • [21] Comment on "Recurrent neural networks: A constructive algorithm, and its properties"
    Personnaz, L
    Dreyfus, G
    NEUROCOMPUTING, 1998, 20 (1-3) : 321 - 324
  • [22] Combining recurrent neural networks and support vector machines for structural pattern recognition
    Jain, BJ
    Geibel, P
    Wysotzki, F
    KI 2004: ADVANCES IN ARTIFICIAL INTELLIGENCE, PROCEEDINGS, 2004, 3238 : 241 - 255
  • [23] Reversible Recurrent Neural Networks
    MacKay, Matthew
    Vicol, Paul
    Ba, Jimmy
    Grosse, Roger
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 31 (NIPS 2018), 2018, 31
  • [24] LEARNING IN RECURRENT NEURAL NETWORKS
    WHITE, H
    MATHEMATICAL SOCIAL SCIENCES, 1991, 22 (01) : 102 - 103
  • [25] Heterogeneous recurrent neural networks
    Lin, JHJ
    Chang, JS
    Chiueh, TD
    IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, 1998, E81A (03) : 489 - 499
  • [26] Recurrent Quantum Neural Networks
    Bausch, Johannes
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 33, NEURIPS 2020, 2020, 33
  • [27] Overview of recurrent neural networks
    Liu J.-W.
    Song Z.-Y.
    Kongzhi yu Juece/Control and Decision, 2022, 37 (11): : 2753 - 2768
  • [28] Pixel Recurrent Neural Networks
    van den Oord, Aaron
    Kalchbrenner, Nal
    Kavukcuoglu, Koray
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 48, 2016, 48
  • [29] RECURRENT NEURAL NETWORKS FOR SYLLABICATION
    HUNT, A
    SPEECH COMMUNICATION, 1993, 13 (3-4) : 323 - 332
  • [30] Stability of Recurrent Neural Networks
    Jalab, Hamid A.
    Ibrahim, Rabha W.
    INTERNATIONAL JOURNAL OF COMPUTER SCIENCE AND NETWORK SECURITY, 2006, 6 (12): : 159 - 164