Structural Properties of Recurrent Neural Networks

被引:1
|
作者
Dobnikar, Andrej [1 ]
Ster, Branko [1 ]
机构
[1] Univ Ljubljana, Fac Comp & Informat Sci, Ljubljana 1000, Slovenia
关键词
Recurrent neural networks; Complex systems; Graph theory; Dynamical systems; ALGORITHM; AUTOMATA;
D O I
10.1007/s11063-009-9096-2
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this article we research the impact of the adaptive learning process of recurrent neural networks (RNN) on the structural properties of the derived graphs. A trained fully connected RNN can be converted to a graph by defining edges between pairs od nodes having significant weights. We measured structural properties of the derived graphs, such as characteristic path lengths, clustering coefficients and degree distributions. The results imply that a trained RNN has significantly larger clustering coefficient than a random network with a comparable connectivity. Besides, the degree distributions show existence of nodes with a large degree or hubs, typical for scale-free networks. We also show analytically and experimentally that this type of degree distribution has increased entropy.
引用
收藏
页码:75 / 88
页数:14
相关论文
共 50 条
  • [41] Locally recurrent neural networks
    Patan, Krzysztof
    Lecture Notes in Control and Information Sciences, 2008, 377 : 29 - 63
  • [42] Bidirectional recurrent neural networks
    Schuster, M
    Paliwal, KK
    IEEE TRANSACTIONS ON SIGNAL PROCESSING, 1997, 45 (11) : 2673 - 2681
  • [43] Discontinuities in recurrent neural networks
    Gavaldà, R
    Siegelmann, HT
    NEURAL COMPUTATION, 1999, 11 (03) : 715 - 745
  • [44] Dilated Recurrent Neural Networks
    Chang, Shiyu
    Zhang, Yang
    Han, Wei
    Yu, Mo
    Guo, Xiaoxiao
    Tan, Wei
    Cui, Xiaodong
    Witbrock, Michael
    Hasegawa-Johnson, Mark
    Huang, Thomas S.
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 30 (NIPS 2017), 2017, 30
  • [45] Flexible Recurrent Neural Networks
    Lambert, Anne
    Le Bolzer, Francoise
    Schnitzler, Francois
    MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES, ECML PKDD 2020, PT I, 2021, 12457 : 694 - 709
  • [46] Improved Prediction of Soil Thermal Properties Using Recurrent Neural Networks
    Slone, Scott Michael
    Zody, Zachary
    Ibey, Robert
    Lein, Wade A.
    INTERNATIONAL CONFERENCE ON TRANSPORTATION AND DEVELOPMENT 2024: TRANSPORTATION SAFETY AND EMERGING TECHNOLOGIES, ICTD 2024, 2024, : 431 - 441
  • [47] Reservoir optimization in recurrent neural networks using properties of Kronecker product
    Rad, Ali Ajdari
    Hasler, Martin
    Jalili, Mandi
    LOGIC JOURNAL OF THE IGPL, 2010, 18 (05) : 670 - 685
  • [48] Synthesis of cascade recurrent neural networks using feedforward generalization properties
    Shaaban, KM
    Schalkoff, RJ
    INFORMATION SCIENCES, 1998, 108 (1-4) : 207 - 217
  • [49] Search of hidden properties in sensor data using recurrent neural networks
    Lemus, L
    López, N
    Barrios, LJ
    Cruz, R
    DATA MINING AND KNOWLEDGE DISCOVERY: THEORY, TOOLS, AND TECHNOLOGY, 1999, 3695 : 92 - 99
  • [50] Convolutional Neural Networks with Recurrent Neural Filters
    Yang, Yi
    2018 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2018), 2018, : 912 - 917