Using Enhanced Simulation Environment to Improve Reinforcement Learning for Satellite Autonomy

What is it about?

Autonomous satellites can decide the best action to take on-the-fly based on their current states without requiring constant human input. Autonomy can allow for better management of large systems of spacecraft while quickly reacting to unexpected events and new requests. Decision-making algorithms for autonomy can be obtained using Deep Reinforcement Learning techniques through repeated interactions with a simulation environment. In a nominal training environment, resource management tasks and unsafe states are infrequent. Enhancing the training environment to make these events more frequent resulted in policies that collected higher rewards and managed better available resources.

Featured Image

Photo by NASA Hubble Space Telescope on Unsplash

Photo by NASA Hubble Space Telescope on Unsplash

Why is it important?

Enhanced training environments can result in policies that can collect more rewards – imaging more targets and/or targets with higher priorities – while showing robust behavior. The robust behavior can be observed by the consistently higher battery energy levels, lower data storage use, and better performance in more challenging environments and longer deployment periods. Consequently, policies trained in such enhanced environments maximize rewards but also showcase enhanced safety attributes, making them more compelling for deployment in real-world missions.