Deep Reinforcement Learning for Joint User Association and Resource Allocation in Factory Automation

We consider the problem of joint user association, channel assignment, and power allocation for mobile robot application in factory automation system that require ultra-reliable and low latency communications (URLLC). The aim is to deliver control commands from the controller to mobile robots with stringent requirements of latency and reliability. To achieve URLLC, we develop a two-phase communication scheme. The robots work close to each other in a factory environment and can form clusters for reliable device-to-device (D2D) communications. Within the latency requirements, the combined payload of a cluster is transmitted to the leader in Phase I. In Phase II, the leader broadcasts the payload to its members. Under this strategy, we use multi-agent reinforcement learning (MARL) for resource allocation. The cluster leaders in Phase I act as the agents and interact with the environment to optimally select the Access Point (AP) for connection along with the sub-band and power level. The objective is to maximize the successful payload delivery probability to all the robots. Illustrative simulation results indicate that the proposed scheme can offer average successful payload delivery probability close to that of centralized exhaustive search algorithm.