Abstract:

A spacecraft multi-stage maneuver decision-making method based on deep reinforcement learning is proposed to address the maneuver decision problem for spacecraft approaching space targets during on-orbit observation service. Firstly, the on-orbit observation task is divided into target approach-observation preparation-continuous observation three stages, establishing the multi-stage task model and constraint set to enhance task solvability. Secondly, the multi-stage policy learning algorithm is proposed, constructing the multi-stage training environment and task reward function, integrating predictive guidance and rule-coupled maneuver guidance mechanisms to enhance algorithm exploration capability and convergence stability. Finally, simulations demonstrate that compared to classical reinforcement learning algorithms, this algorithm reduces convergence time by 30.9%, increases average task cumulative reward by 9.28%, and decreases average pulse consumption by 13.91%. Moreover, compared to the traditional optimization method, it effectively enhances core task indicators, validating its effectiveness.

Key words: spacecraft maneuvering, on-orbit observation, intelligent decision-making, deep reinforcement learning