Probabilistic Bayesian Neural Networks for Efficient Inference

Bayesian Neural Networks (BNNs) offer a principled framework for modeling uncertainty in deep learning tasks. However, conventional BNNs often suffer from high computational complexity and parameter overhead. In this paper, we propose a novel approach, termed Probabilistic BNN (ProbBNN), which leverages probabilistic computing principles to streamline the inference process. Unlike traditional deterministic approaches, ProbBNN represents inputs and parameters as random variables governed by probability distributions, allowing for the propagation of uncertainty throughout the network. We employ Gaussian Mixture Models (GMMs) to represent the parameters of each neuron or convolutional kernel, enabling efficient encoding and processing of uncertainty. Our approach simplifies the inference process by replacing complex deterministic computations with lightweight probabilistic operations, resulting in reduced computational complexity and improved scalability. Experimental results demonstrate the effectiveness of ProbBNN in achieving competitive accuracy to traditional BNNs while significantly reducing the number of parameters. The transition to ProbBNN yields a reduction of two orders of magnitude in the number of parameters compared to baseline approaches, making our approach promising for deployment in resource-constrained applications such as edge computing and IoT devices.