Mapping some basic functions and operations to multilayer feedforward neural networks for modeling nonlinear dynamical systems and beyond

This study significantly extends the development of an initialization methodology for designing multilayer feedforward neural networks, aimed primarily at modeling nonlinear functions for engineering mechanics applications, as proposed and published in (Pei in Ph.D. dissertation, Columbia University, 2001; Pei and Smyth in J. Eng. Mech. 132(12):1290, 1310, 2006; Pei et al. in Comput. Methods Appl. Mech. Eng. 194(42-44):4481, 2005; Pei et al. in Proc. Int. Joint Conference on Neural Networks (IJCNN'05), pp. 1377-1382, 2005; Pei and Mai in J. Appl. Mech. 2008; Pei et al. in Proc. Int. Joint Conference on Neural Networks (IJCNN'07), 2007). Seeking a transparent and domain knowledge-based approach for neural network initialization and result interpretation, this study examines linear sums of sigmoidal functions as a means to construct approximations to various nonlinear functions including reciprocal, absolute value, the product of absolute value and first-order polynomial, exponential, truncated sinc, Mexican hat, and Gaussian functions as well as the four elementary arithmetic operations (addition, subtraction, multiplication, and division). By extending two initialization techniques (layer condensation and inspiration from high-order derivatives of sigmoidal function), this study advances the previously proposed initialization procedure, thus opening the door to a significantly wider range of nonlinear functions. Specifically, in engineering mechanics, this study directly benefits multilayer feedforward neural networks when modeling nonlinear restoring forces based on the force-state mapping (among others). Application examples are provided to illustrate the importance of studying basic functions and operations, and future work is identified.