Applying Piecewise Linear Approximation for DNN Non-Linear Activation Functions to Bfloat16 MACs

The efficient implementation of inference engines for the DNN (Deep Neural Network) execution requires low power consumption and a small area. However, the activation functions' implementation is challenging since they require considerable computing resources due to their non-linearity. In this paper, we study the non-linearity of the functions and show that only one MAC execution using our PLA (Piecewise Linear Approximation) scheme is sufficient to guarantee the accuracy in bfloat16. For the evaluation, we applied our proposal to our in-house bfloat16 MACs and achieved that our results were less than 1 LSB difference from ideal values on average.