Secure and Efficient Federated Learning by Combining Homomorphic Encryption and Gradient Pruning in Speech Emotion Recognition

Speech Emotion Recognition (SER) detects human emotions expressed in spoken language. SER is highly valuable in diverse fields; however, privacy concerns arise when analyzing speech data, as it reveals sensitive information like biometric identity. To address this, Federated Learning (FL) has been developed, allowing models to be trained locally and just sharing model parameters with servers. However, FL introduces new privacy concerns when transmitting local model parameters between clients and servers, as third parties could exploit these parameters and disclose sensitive information. In this paper, we introduce a novel approach called Secure and Efficient Federated Learning (SEFL) for SER applications. Our proposed method combines Paillier homomorphic encryption (PHE) with a novel gradient pruning technique. This approach enhances privacy and maintains confidentiality in FL setups for SER applications while minimizing communication and computation overhead and ensuring model accuracy. As far as we know, this is the first paper that implements PHE in FL setup for SER applications. Using a public SER dataset, we evaluated the SEFL method. Results show substantial efficiency gains with a key size of 1024, reducing computation time by up to 25% and communication traffic by up to 70%. Importantly, these improvements have minimal impact on accuracy, effectively meeting the requirements of SER applications.