The Application of Neural Networks for Facial Landmarking on Mobile Devices

Many modern mobile applications incorporate face detection and landmarking into their systems, such as Snapchat, beauty filters and camera auto-focusing systems, where they implement regression based machine learning algorithms for accurate face landmark detection, allowing the manipulation of facial appearance. The mobile applications that incorporate machine learning have to overcome issues such as lighting, occlusion, camera quality and false detections. A solution could be provided through the resurgence of deep learning with neural networks, as they are showing significant improvements in accuracy and reliability in comparison to the state-of-the-art machine learning. Here, we demonstrate the process by using trained networks on mobile devices and review its effectiveness. We also compare the effects of employing max-pooling layers, as an efficient method to reduce the required processing power. We compared network with 3 different amounts of max-pooling layer and ported one to the mobile device, the other two could not be ported due to memory restrictions. We will be releasing all code to build, train and use the model in a mobile application. The results show that despite the limited processing capability of mobile devices, neural networks can be used for difficult challenges while still working in real-time. We show a network running on a mobile device on a live data stream and give a recommendation on the structure of the network.