Secure Neural Network Inference as a Service with Resource-Constrained Clients

Applying services computing to neural networks, a service provider may provide inference with a pre-trained neural network as a service. Clients use the service to get the neural network's output on their input. To protect sensitive data, secure neural network inference (SNNI) entails that only the client learns the output; the input remains the client's secret and the neural network's parameters remain the service provider's secret. Several SNNI approaches were proposed and evaluated in environments where both service providers and clients used powerful computers. In many real settings, for instance in edge computing, client devices are resource-constrained. This paper is the first to investigate the impact of client-side resource constraints on SNNI. We perform experiments with two state-of-the-art SNNI approaches and three neural networks. We vary the compute and memory capacity of the client device and measure the impact on inference time. Our findings show that client-side resource constraints significantly impact the performance and even the applicability of SNNI approaches. The results indicate the limits of current SNNI approaches for resource-constrained clients. Based on the results, we identify research directions to improve SNNI for resource-constrained clients.