A novel scheme to determine the architecture of a Multi-layer Perceptron

We propose a method for optimizing the architecture, of a Multilayer Perceptron (MLP) network. The proposed scheme is a variation of the MLP architecture, in which each neuron's output is modulated by an efficiency factor associated with that node. Nodes with low efficiencies (efficiency factors) literally do not participate in the network. We compute the efficiency of a node using a multiplier function with a learnable parameter which we call as the multiplier of that node. Values of the multipliers are learned by a gradient descent along with the weights, aiming to minimize the mean square error (MSE). Training starts with all node efficiencies set very low so that there is literally no connection between any of the neurons in the net. As the learning progresses, gradually some of the nodes start acquiring high efficiencies. Training is terminated when performance of the network is satisfactory. At the end of the training, nodes with low efficiency are eliminated and a near optimal architectural size fdr the MLP is obtained. Effectiveness of the proposed scheme is demonstrated on several data-sets.