A Communication-Efficient Federated Learning by Dynamic Quantization and Free-Ride Coding

This paper focuses on the design of dynamic quantization (DQ) and coded transmission schemes for federated learning (FL). In the conventional FL system, the updates divergences typically decrease with communication rounds as training goes on. We first study both the impact of the quantization bit-width in the error-free transmission scenario and the impact of bit error rate (BER) in the practical transmission scenario on the performance of FL. Then we propose a DQ scheme based on the fixed B-bit or 1-bit quantization scheme, where each device quantizes its local updates with dynamic bit-width according to the test accuracy of the global model and device-to-server signal-to-noise ratio (SNR). Due to the quantization bit-width is dynamic, the test accuracy (as a kind of extra data) is needed for devices to determine the bit-width, and the devices need to inform the server of the resultant bit-width (as another kind of extra data). To reliably transmit these extra data without consuming extra transmission resource, we utilize the free-ride coding, where the extra data are embedded into the low-density parity-check (LDPC) coded payload data. Numerical results show that in the practical scenario, B-bit (B > 1) quantization scheme shows fast convergence speed and high final accuracy (in high SNR region) while the 1-bit quantization scheme exhibits greater robustness (in low SNR region). They also show that the proposed FL by DQ and free-ride coding not only can significantly reduce the communication overhead with a negligible performance gap to the upper bound (error-free scheme) even in low SNR region but also outperforms the fixed quantization coded transmission FL scheme in terms of accuracy and convergence speed.