Federated Classifier Fusion - An Evidential Reasoning Approach

Federated Learning (FL) aims to solve the problem of data privacy in modern machine learning systems by training a local model at each data silo and only sharing model updates with the server to aggregate parameters to learn a global model. Despite its recent success, various studies have mentioned high variance in the performance and vulnerability to overfitting in FL algorithms. As an effort to mitigate this issue, we adopted the idea of the stacking-based ensemble to FL to reduce generalization error. In ensemble learning, the choice of combination method directly impacts the performance of the ensemble model. We evaluated the Evidential Reasoning (ER) rule as a combiner, or meta-learner, algorithm. The ER rule can combine multiple pieces of independent and highly conflicting evidence with different weights and reliability, and it is used to fuse predictions of multiple federated classifiers in this work. Empirical results showed that the stacking-based ER ensemble can achieve lower loss than its component learners and outperform alternative combination methods. These results suggest that the ER rule can serve as a reliable choice or a baseline combination method for stacking-based ensemble learning of federated classifiers.