Importance-Aware Message Exchange and Prediction for Multi-Agent Reinforcement Learning

The cooperation among intelligent agents is the key to build smarter real-world intelligent systems. Recent researches have shown that wireless communication plays a vital role in multi-agent cooperation. However, limited wireless resources become the bottleneck of large-scale multi-agent cooperation. Therefore, we focus on how to improve the performance of multi-agent reinforcement learning under the constraint of communication resources. To share more valuable information among agents under resource constraint, we formulate the message importance and design a decentralized scheduling policy with query mechanism, so that agents can effectively exchange messages according to their message importance. We further design a message prediction mechanism to compensate for those messages that are not scheduled for transmission in the current round. Finally, we evaluate the performance of the proposed schemes in a traffic junction environment, where only a fraction of agents can broadcast their messages in each round due to limited wireless resources. Results show that the importance-aware message exchange can extract valuable information to guarantee the system performance even when less than half of agents can share their states. By exploiting message prediction, the system can further save 40% of communication resources while guaranteeing the system performance.