Learning Properties of Feedforward Neural Networks Using Dual Numbers

Real-valued neural networks (real-NN) with real numbers is a popular neural network model. Complex-valued neural network (complex-NN) is an extension of the real-NN to the complex domain where the inputs, outputs, weights, and biases are all complex numbers. Dual number is a two-dimensional number and is kind of a relative of complex numbers. In this paper, we investigate a feedforward neural network extended to dual numbers (dual-NN). It is found that the dual-NN has different properties from the real-NN and the complex-NN. Specifcally, for the input (two-dimensional information), the weights are trained with the constraint of the motion of shearing. Experimental results show that the generalization ability is higher than those of the real-NN and the complex-NNs for the training pattern with the shearing relation.