FIRMLP for Handwritten Digit Recognition

The finite impulse response multilayer perceptron (FIRMLP), a class of temporal processing neural networks, is a multilayer perceptron where the static weights (synapses) have been replaced by finite impulse response filters. Thus FIRMLPs are a type of convolutional neural network and different synapse types can be considered. We compare the performance of different network configurations for the recognition task by using the MNIST database. Different fully or partially connected neural networks configurations have been created by varying the number of hidden layers, the number of neurons and their synapse type. These simple architectures combined with a pattern selection algorithm based on error threshold achieve state-of-the-art recognition accuracy. Partially connected FIRMLPs containing as few as 300 neurons achieve recognition rates of about 0.8%. The FIRMLPs are easy to train by showing fast convergence. Networks with strong delay synapses are robust to overfitting as well. Our proposed aproach composed of an ensemble of FIRMLPs with different synapse types has demonstrated the state-of-the-art classification performance by winning the Handwritten Digit Recognition Competition (HDRC 2013) organized within ICDAR 2013.