How Deep Is Optimal for Learning Locally on Smartphone

In this paper, we introduced some methods to optimize such deep learning models as convolutional neural network and recurrent neural network. Firstly, we introduced some dropout techniques. By regularizing with dropout, we will prevent overfitting and balance the depth of the neural network and the width of each layer. Secondly, we designed an architecture with less layers but more sophisticated activation functions. Thirdly, we adjusted the learning rate and momentum of stochastic gradient descent (SGD) optimization algorithm. SGD can lead to fast convergence by following the negative gradient of the objective based on a mini batch of training subset. Finally, we discussed model compression. The aim is to train locally the deep learning model on smartphone.