Deep Reinforcement Learning for Decentralized Autonomous Decision-Making in Federated Satellite Systems

The exponential growth in the number of satellites orbiting Earth is in need of the development of more efficient, autonomous decision-making frameworks for managing large satellite constellations. Traditional centralized methods of satellite operation are increasingly inadequate in dealing with the dynamic and unpredictable nature of space environments. To address these challenges, this thesis investigates the application of Deep Reinforcement Learning (DRL) for decentralized autonomous decision-making in Federated Satellite Systems (FSS), a paradigm that enables the collaborative use of satellite resources across different owners and missions. This research presents a comprehensive framework that integrates DRL into satellite operations, with the objective of enhancing real-time decision-making capabilities. A modular simulation environment was developed to model the interactions between multiple satellites within an FSS, simulating various operational scenarios including resource sharing and communication management. The simulation framework supports diverse satellite types and coordination models, ranging from fully centralized to fully decentralized configurations, allowing for a thorough evaluation of different operational strategies. Three state-of-the-art DRL algorithms—Deep Q-Networks (DQN), Soft Actor-Critic (SAC), and Proximal Policy Optimization (PPO)—were implemented and trained within this simulation environment. These algorithms were evaluated based on their performance in optimizing satellite operations, particularly in tasks such as target observation, data sharing, and resource management. The scenario considered for this project consists of 20 observer satellites and 20 target objects orbiting Earth. The main goal for the observers is to detect and observe the targets while in orbit, and meanwhile spread the obtained data between all participants while leveraging energy and storage resources. The comparative analysis of these AI agents in different coordination models highlights the strengths and weaknesses of each algorithm in different operational contexts, providing valuable insights into the trade-offs between computational efficiency, scalability, and mission success. Furthermore, the practical feasibility of deploying these DRL algorithms in real satellite systems was assessed by implementing them on an NVIDIA Jetson, a hardware platform representative of onboard satellite computers. The performance benchmarks indicate that DRL-based approaches can be effectively integrated into existing satellite architectures, paving the way for more autonomous and resilient space missions.     «

The exponential growth in the number of satellites orbiting Earth is in need of the development of more efficient, autonomous decision-making frameworks for managing large satellite constellations. Traditional centralized methods of satellite operation are increasingly inadequate in dealing with the dynamic and unpredictable nature of space environments. To address these challenges, this thesis investigates the application of Deep Reinforcement Learning (DRL) for decentralized autonomous decisio...     »