Ftmoe: a federated transfer model based on mixture-of-experts for heterogeneous image classification

In recent years, significant advancements in machine learning have led to an urgent demand for data privacy protection. The emergence of federated learning (FL) has offered a new direction to address this issue. However, two critical challenges hinder the widespread adoption of FL. Firstly, the global model cannot perform well on each participant’s task when facing data distribution heterogeneity. Secondly, the traditional FL system’s over-reliance on a single center affects its security and credibility. In this article, we propose a distributed federated transfer model based on the Mixture-of-Experts (MoE) technique for personalized image classification. During local training, we design a transfer learning (TL) method based on MoE that constructs relatively customized models according to the comprehensive opinions of each expert for different participants. The MoE utilizes a gated network to assign the contributions of various experts through a set of weight values. We novelly combine expert opinions with the original analysis to enable models to focus on local data features. In addition, we exploit blockchain technology to achieve secure data storage in FL framework. The smart contracts and a consensus algorithm based on node quality efficiently prevent data leakage and tampering. The results show that FTMoE outperforms conventional schemes in terms of personalized classification accuracy, which ranges from 0.4% to 4%, while maintaining system credibility.