Evaluating a Reinforcement Learning Approach for Collision Avoidance with Heterogeneous Aircraft

This paper focuses on the problem of decentralized collision avoidance for Unmanned Aircraft Systems. The considered setting involves drones that can sense the presence of other traffic within a specified sensing area, for example due to remote ID signal broadcasts or visual perception, and take evasive maneuvers according to drone dynamics to ensure a minimum separation. A Reinforcement Learning approach is investigated to adapt the ego drone trajectory in response to the limited observations of the intruders trajectory within a bounded environment. The key aim of the work is on designing evasive maneuvers for ego drone when sharing the airspace with heterogeneous aircraft that have varying sensing capabilities, maneuverability, and risk-awareness. Beside reachability-based techniques, which offer a powerful framework for identifying safe trajectories under worst case actions by other agents, the proposed approach aims at adapting the evasive maneuver to the incoming aircraft behavior. Tests were executed in the simulated environment comparing the results obtained with heterogeneous incoming aircraft with different maneuverability levels and with randomly selected fixed obstacles. The Reinforcement Learning based method performance was also tested adopting different state vector parameters. The results show that the proposed approach can support the implementation of safe collision avoidance services allowing the generation of adaptive maneuvers for Unmanned Aerial System Traffic Management.