Optimizing Future's Dense Air Traffic via Deep Reinforcement Learning

What is it about?

Urban Air Mobility (UAM) and Unmanned Aircraft Systems (UAS) are anticipated to result in a significant growth in air traffic. Novel solutions are required to solve the problems of future's air traffic such as path planning with separation constraints. In this paper, we apply MuZero, a newly-introduced deep reinforcement learning algorithm by DeepMind to path planning problems in dynamic air traffic environments. To formulate the path planning problem, we consider a sequential trajectory allocation approach that would act on a "first-come-first-serve" basis. Initial results show that agents can learn to mitigate collisions when trained with the obstacle avoidance framework without requiring any knowledge about the domain and rules.

Featured Image

Photo by Luke Jernejcic on Unsplash

Photo by Luke Jernejcic on Unsplash

Why is it important?

The dense urban airspace associated with air taxi and aerial package delivery services will pose challenges to managing the future's airspace containing agents having different flight characteristics and procedures. In this work, we investigated the MuZero algorithm, a recent deep reinforcement learning method as a potential approach to solve such air traffic problems. Since the algorithm does not require knowledge about the domain, expert data, or airspace rules, it is beneficial for creating policies in unknown environments. The multi-agent path planning problem is formulated as a turn-based sequential path planning problem analogous to the concept of turn-based games. We demonstrated that it is possible to create policies for obstacle avoidance in a game setting without requiring knowledge of the domain or rules.