TY - JOUR

T1 - A multi-aircraft co-operative trajectory planning model under dynamic thunderstorm cells using decentralized deep reinforcement learning

AU - Pang, Bizhao

AU - Xu, Xinting

AU - Zhang, Mincheng

AU - Alam, Sameer

AU - Lulli, Guglielmo

PY - 2025/2/3

Y1 - 2025/2/3

N2 - Climate change induces an increased frequency of adverse weather, particularly thunderstorms, posing significant safety and efficiency challenges in en route airspace, especially in oceanic regions with limited air traffic control services. These conditions require multi-aircraft cooperative trajectory planning to avoid both dynamic thunderstorms and other aircraft. Existing literature has typically relied on centralized approaches and single agent principles, which lack coordination and robustness when surrounding aircraft or thunderstorms change paths, leading to scalability issues due to heavy trajectory regeneration needs. To address these gaps, this paper introduces a multi-agent cooperative method for autonomous trajectory planning. The problem is modeled as a Decentralized Markov Decision Process (DEC-MDP) and solved using an Independent Deep Deterministic Policy Gradient (IDDPG) learning framework. A shared actor-critic network is trained using combined experiences from all aircraft to optimize joint behavior. During execution, each aircraft acts independently based on its own observations, with coordination ensured through the shared policy. The model is validated through extensive simulations, including uncertainty analysis, baseline comparisons, and ablation studies. Under known thunderstorm paths, the model achieved a 2 % loss of separation rate, increasing to 4 % with random storm paths.ETA uncertainty analysis demonstrated the model’s robustness, while baseline comparisons with the Fast Marching Tree and centralized DDPG highlighted its scalability and efficiency. These findings contribute to advancing autonomous aircraft operations.

AB - Climate change induces an increased frequency of adverse weather, particularly thunderstorms, posing significant safety and efficiency challenges in en route airspace, especially in oceanic regions with limited air traffic control services. These conditions require multi-aircraft cooperative trajectory planning to avoid both dynamic thunderstorms and other aircraft. Existing literature has typically relied on centralized approaches and single agent principles, which lack coordination and robustness when surrounding aircraft or thunderstorms change paths, leading to scalability issues due to heavy trajectory regeneration needs. To address these gaps, this paper introduces a multi-agent cooperative method for autonomous trajectory planning. The problem is modeled as a Decentralized Markov Decision Process (DEC-MDP) and solved using an Independent Deep Deterministic Policy Gradient (IDDPG) learning framework. A shared actor-critic network is trained using combined experiences from all aircraft to optimize joint behavior. During execution, each aircraft acts independently based on its own observations, with coordination ensured through the shared policy. The model is validated through extensive simulations, including uncertainty analysis, baseline comparisons, and ablation studies. Under known thunderstorm paths, the model achieved a 2 % loss of separation rate, increasing to 4 % with random storm paths.ETA uncertainty analysis demonstrated the model’s robustness, while baseline comparisons with the Fast Marching Tree and centralized DDPG highlighted its scalability and efficiency. These findings contribute to advancing autonomous aircraft operations.

KW - Air traffic management

KW - Autonomous trajectory planning

KW - Multi-aircraft coordination

KW - Deep reinforcement learning

KW - Dynamic thunderstorm cells

KW - Climate change

U2 - 10.1016/j.aei.2025.103157

DO - 10.1016/j.aei.2025.103157

M3 - Journal article

VL - 65

JO - Advanced Engineering Informatics

JF - Advanced Engineering Informatics

SN - 1474-0346

M1 - 103157

ER -