Abstract:

In recent years, the threat of unmanned underwater vehicles (UUV) to national sea security has gradually increased. At the same, it is difficult to detect UUVs for its low noise and team intrusion. A two stage anti-submarine planning method is proposed to learn the optimal anti-submarine strategy. During the deployment stage, a resource allocation model based on uncertain Markov decision process (MDP) is proposed, whose Nash equilibrium point is solved by the elaborately designed robust reinforcement learning algorithm of the deployment strategy. In the search stage, a search model based on partially observable Markov decision process (POMDP) is proposed which is solved by the search strategy learning algorithm based on multi-agent reinforcement learning (MARL). Simulation results show that the proposed algorithm outperforms other algorithms.

Key words: anti-submarine, unmanned underwater vehicle (UUV), multi-agent reinforcement learning, game theory