RLink: Accelerate On-Device Deep Reinforcement Learning with Inference Knowledge at the Edge

Deep reinforcement learning (DRL) has been a successful paradigm in machine learning that enables solving complex control problems at the human level. However, the sampling and training efficiency of state-of-the-art DRL frameworks can not satisfy the stringent latency and throughput requirements of today's mobile environments. Existing distributed and offline reinforcement learning algorithms along with the libraries for training acceleration are inherently designed for DRL tasks performed in the cloud rather than on distributed mobile devices, on which the computing resources are highly constrained, heterogeneous, and possibly dynamically changing. With the rise of edge computing and intelligence services, this paper presents RLink, a novel distributed training library to accelerate on-device deep reinforcement learning with inference knowledge at the edge. We leverage knowledge distillation to realize lightweight interaction between our on-device training task and the remote models that can provide inference knowledge. In this way, RLink is designed to be event-driven and agnostic to heterogeneous deep reinforcement learning algorithms and libraries. To tackle the communication bottleneck, a novel asynchronous sampling algorithm is proposed to facilitate real-time training in RLink. Tuned for unstable-connected mobile devices, RLink is robust and efficient by using a semantic-aware communication pipeline for lossless data compression. Extensive experimental results show that, compared with state-of-the-art algorithms and libraries, RLink can accelerate deep reinforcement learning at the edge with up to decuple speedups in convergence and ideal computational performance.