Noise Resilience of Reduced Precision Neural Networks

Reduced Precision Neural Networks, where computations are performed with as low as one or two bits of precision, are starting to find relevance in a wide range of applications, including vision, speech, and natural language processing. Such networks are capable of running on low power and cost on embedded systems such as FPGAs (field programmable gate arrays). Recent research has extensively studied and advanced the accuracy of these networks. However, unlike regular neural networks, little is known about how resilient they are in the presence of noisy input data. From old photographs that are rediscovered when you dig through your attic to images taken from thousands of miles away in space, noisy input data is a common factor in everyday life. In this study, we characterize the behavior of Reduced precision neural networks to noisy input data and identify techniques to improve their resilience. Benchmark image data is injected with different noise profiles, and the inference capabilities of reduced-precision networks (based on Yolo, Dorefa-net) are studied and contrasted with full precision neural networks. Experimental results show that reduced-precision networks perform well, within 1-5% accuracy, relative to full precision networks in the presence of significant levels of noise. We also show that significant improvements (> 20%) to overall image recognition accuracy are possible to achieve by creating a high-quality ensemble neural network, which combines multiple reduced-precision neural networks.