Efficient Sparse Networks from Watts-Strogatz Network Priors

This paper studies the accuracy and the structural properties of sparse neural networks (SNNs) generated by weight pruning and by using Watts-Strogatz network priors. The study involves Multi-Layer Perceptron (MLP) and Long-Short Term Memory (LSTM) architectures, trained on the MNIST dataset. The paper replicates and extends previous work, showing that networks generated by appropriately selected WS priors guarantee high-quality results, and that these networks outperform pruned networks in terms of accuracy. In addition, observations are made with regard to the structural change induced by network pruning and its implications for accuracy. The findings of this study provide important insights for creating lighter models with lower computational needs, which can achieve results comparable to more complex models.