Fine-Grained Offloading for Multi-Access Edge Computing with Actor-Critic Federated Learning

In this paper, we study fine-grained offloading for multi-access edge computing (MEC) in 5G. Existing works for computation offloading is on a per-task basis and do not take into account the execution order among tasks in one application. Fine-grained offloading, on the other hand, considers the task structure of an application upon making offloading decision and may only offload computation-hungry tasks to the MEC, thus making better use of system resource. To solve the problem, we propose an online solution based on Actor-Critic Federated Learning, called AC-Federate. In AC-Federate, we consider a multi-MEC network in which each edge node trains a model-free advantage Actor-Critic (AC) model based on local data. The AC model of each edge node jointly optimizes the continuous actions (i.e., radio and computing resource allocations) and the discrete action (i.e., offloading decision), and trains the model with a weighted loss function. To further improve the inference accuracy of the AC model, each edge node uploads the gradients of its actor and critic neural networks to a central controller in an asynchronous manner. The central controller then ensembles the collected gradients from different edge nodes and updates all edge nodes with the integrated network parameters. Simulation results show that the proposed AC-Federate outperforms DDPG and others in terms of delay, energy consumption, and mixed consideration of delay and energy consumption performance even when the number of UEs is very large.