Multi-Agent and Multi-Target Reinforcement Learning for Satellite Sensor Tasking

Advancements in machine learning and artificial intelligence have found broad applications across various domains. However, their integration into space-related scenarios, especially satellite tasking, has been relatively limited. As satellite architectures shift from a few high-end systems to constellations of simpler, more numerous spacecraft, dynamically tasking payload sensors becomes intricate. This complexity underscores the importance of researching effective sensor tasking methods, a challenge ideally suited for Reinforcement Learning (RL). This study expands upon prior work in RL for Low Earth Orbit (LEO) satellite Hyper Spectral Imaging (HSI) sensor tasking, broadening its scope to encompass multi-agent, multi-target scenarios. To enable the effective use of RL in multi-agent systems, we propose an enhanced Python-based modeling and simulation environment. This framework facilitates the development and assessment of coordination and decision-making processes among multiple independent satellites acting as agents in the LEO domain. Through the incorporation of multi-agent reinforcement learning (MARL) methodologies, the framework fosters collaboration and communication among satellites while optimizing their individual tasking goals. Additionally, we extend the RL framework to support the simultaneous tracking of multiple targets. This involves adapting existing algorithms to accommodate multi-target reinforcement learning (MTRL) strategies, where each satellite agent dynamically allocates its sensing resources to diverse targets based on relevance and importance. This work showcases a computationally efficient extension from a single-agent, single-target approach to a multi-agent, multi-target context. We outline steps to streamline the multi-target association into a single-target context, thus expanding RL capabilities with minimal added computational complexity.