A Hybrid Deep Reinforcement Learning Approach for Dynamic Task Offloading in NOMA-MEC System

Mobile edge computing (MEC) has been regarded as a promising paradigm for increasing the computing capacity of mobile devices (MDs) by offloading tasks to edge servers. Non-orthogonal multiple access (NOMA) is a critical multiple access technique that allows many MDs to transmit on the same resource block simultaneously. Attracted by the enormous benefits of combining NOMA and MEC, we investigate the dynamic computation offloading in a multi-device multi-server NOMA-MEC system. We consider the partial offloading policy such that MDs can offload a portion of the task to the edge server for execution. To minimize the overall computation delay and energy consumption, we formulate a mixed integer programming (MIP) problem to jointly optimize edge server selection and offloading task ratio. Solving the optimization problem with a discrete-continuous hybrid action space is not straightforward since most existing deep reinforcement learning (DRL) algorithms are only applicable to discrete or continuous action spaces. We present the hybrid advantage actor-critic (HA2C) approach, which employs an actor-critic architecture consisting of two parallel actor networks and a critic network, to tackle this problem. Specifically, the discrete actor and continuous actor networks based on deep neural networks (DNNs) determine MEC server selection and offloading ratio, respectively. The critic network evaluates the current state value, and the advantage function is computed for the discrete and continuous network parameter updates. Experimental results show that the proposed algorithm is superior to conventional DRL algorithms that convert the hybrid action space into a unified homogeneous action space.