基于强化学习的海上移动目标搜索路径规划

doi:10.12305/j.issn.1001-506X.2026.02.13

Abstract

Abstract:

Maritime moving target search plays a crucial role in maritime search and rescue operations, as the search efficiency directly impacts the success rate of the operations. Aiming at the maritime moving target search path planning problem, the following three parts of work are done. Firstly, an algorithm to determine the search area based on the minimum coverage rectangle is proposed to divide the optimal search area. Next, a search path planning algorithm with reinforcement learning is constructed, comprehensively considering multiple constraints, and achieving optimal search path planning of sub-areas. Finally, a decision support system is developed to verify the effectiveness and robustness of the algorithms and it is extended to scenarios involving multi-search unit cooperation. The results demonstrate that the proposed algorithms have achieved positive outcomes across a spectrum of search scenarios. This showcases not only considerable theoretical significance but also practical potential in enhancing the success rate of search and rescue operations.

Key words: maritime moving targets search, reinforcement learning, path planning, decision support

CLC Number:

N 945.15

Pengcheng YANG, Qingqing YANG, Yingying GAO, Zhiwei YANG, Kewei YANG, Bo AI. Path planning for maritime moving targets search based on reinforcement learning[J]. Systems Engineering and Electronics, 2026, 48(2): 515-523.

Figures/Tables 20

Table 1

Fig.1

Table 2

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Table 3

Fig.14

Fig.15

Fig.16

Fig.17

References 32

1	ZHENG L X, WU M Q, ZHAO J, et al. Effects of Ulva prolifera dissipation on the offshore environment based on remote sensing images and field monitoring data[J]. Acta Oceanologica Sinica, 2023, 42 (6): 112- 120. doi: 10.1007/s13131-022-2129-7
2	YANG D, YUEN K V, GU X F, et al. Influences of environmental factors on the dissipation of green tides in the Yellow Sea, China[J]. Marine Pollution Bulletin, 2023, 189, 114737. doi: 10.1016/j.marpolbul.2023.114737
3	LIU Z K, HAN Z H, CHEN Q, et al. Risk assessment of marine oil spills using dynamic Bayesian network analyses[J]. Environmental Pollution, 2023, 317, 120716. doi: 10.1016/j.envpol.2022.120716
4	LI J Q, ZHANG G Q, JIANG C Y, et al. A survey of maritime unmanned search system: theory, applications and future directions[J]. Ocean Engineering, 2023, 285, 115359. doi: 10.1016/j.oceaneng.2023.115359
5	KOOPMAN B O. The theory of search. I. kinematic bases[J]. Operations Research, 1956, 4 (3): 324- 346. doi: 10.1287/opre.4.3.324
6	KOOPMAN B O. The theory of search. II. target detection[J]. Operations Research, 1956, 4 (5): 503- 531. doi: 10.1287/opre.4.5.503
7	KOOPMAN B O. The theory of search: III. the optimum distribution of searching effort[J]. Operations Research, 1957, 5 (5): 613- 626. doi: 10.1287/opre.5.5.613
8	杨克巍, 梁笑天, 郭玙, 等. 海上搜寻理论方法与搜救决策支持系统现状及发展[J]. 海岸工程, 2021, 40 (4): 291- 302. doi: 10.3969/j.issn.1002-3682.2021.04.006
	YANG K W, LIANG X T, GUO Y, et al. Status and development of maritime search theory and search and rescue decision support system[J]. Coastal Engineering, 2021, 40 (4): 291- 302. doi: 10.3969/j.issn.1002-3682.2021.04.006
9	BREIVIK O, ALLEN A A. An operational search and rescue model for the Norwegian Sea and the North Sea[J]. Journal of Marine Systems, 2008, 69 (1/2): 99- 113. doi: 10.1016/j.jmarsys.2007.02.010
10	AN D D, ZHANG T C, ZHANG J, et al. Research on drift path prediction technology of ships[C]//Proc. of the International Conference on Computer Modeling, Simulation and Algorithm, 2018: 231−235.
11	BURCIU Z. Application of Fokker-Planck equation for modeling the search and rescue area at sea[J]. Annual of Navigation, 2002, 4, 21- 32.
12	OTOTE D A, LI B, AI B, et al. A decision-making algorithm for maritime search and rescue plan[J]. Sustainability, 2019, 11 (7): 2084. doi: 10.3390/su11072084
13	ARKIN E M, FEKETE S P, MITCHELL J S B. Approximation algorithms for lawn mowing and milling[J]. Computational Geometry, 2000, 17 (1/2): 25- 50. doi: 10.1016/S0925-7721(00)00015-8
14	HERMANS B, LEUS R, MATUSCHKE J. Exact and approximation algorithms for the expanding search problem[J]. INFORMS Journal on Computing, 2022, 34 (1): 281- 296. doi: 10.1287/ijoc.2020.1047
15	MORIN M, ABI-ZEID I, QUIMPER C G. Ant colony optimization for path planning in search and rescue operations[J]. European Journal of Operational Research, 2023, 305 (1): 53- 63. doi: 10.1016/j.ejor.2022.06.019
16	LI X W, GAO M, KANG Z, et al. Collaborative search and rescue based on swarm of H-MASSs using consensus theory[J]. Ocean Engineering, 2023, 278, 114426. doi: 10.1016/j.oceaneng.2023.114426
17	DAH-ACHINANON U, BAJESTANI S E M, LAJOIE P Y, et al. Search and rescue with sparsely connected swarms[J]. Autonomous Robots, 2023, 47 (7): 849- 863. doi: 10.1007/s10514-022-10080-7
18	LAPORTE G, LOUVEAUX F, MERCURE H. Models and exact solutions for a class of stochastic location-routing problems[J]. European Journal of Operational Research, 1989, 39 (1): 71- 78. doi: 10.1016/0377-2217(89)90354-8
19	AGNETIS A, HERMANS B, LEUS R, et al. Time-critical testing and search problems[J]. European Journal of Operational Research, 2022, 296 (2): 440- 452. doi: 10.1016/j.ejor.2021.03.038
20	李苯帅. 基于最优搜寻理论的海上搜救方案规划方法研究[D]. 青岛: 山东科技大学, 2020.
	LI B S. Research on planning method of maritime search and rescue based on optimal search theory[D]. Qingdao: Shandong University of Science and Technology, 2020.
21	CHO S W, PARK H J, LEE H, et al. Coverage path planning for multiple unmanned aerial vehicles in maritime search and rescue operations[J]. Computers & Industrial Engineering, 2021, 161, 107612.
22	SUTTON R S, BARTO A G. Reinforcement learning: an introduction[M]. Cambridge: MIT Press, 2018.
23	AI B, JIA M X, XU H W, et al. Coverage path planning for maritime search and rescue using reinforcement learning[J]. Ocean Engineering, 2021, 241, 110098. doi: 10.1016/j.oceaneng.2021.110098
24	WOO J, YU C, KIM N. Deep reinforcement learning-based controller for path following of an unmanned surface vehicle[J]. Ocean Engineering, 2019, 183, 155- 166. doi: 10.1016/j.oceaneng.2019.04.099
25	杨清清, 高盈盈, 郭玙, 等. 基于深度强化学习的海战场目标搜寻路径规划[J]. 系统工程与电子技术, 2022, 44 (11): 3486- 3495. doi: 10.12305/j.issn.1001-506X.2022.11.24
	YANG Q Q, GAO Y Y, GUO Y, et al. Target search path planning for naval battle field based on deep reinforcement learning[J]. Systems Engineering and Electronics, 2022, 44 (11): 3486- 3495. doi: 10.12305/j.issn.1001-506X.2022.11.24
26	高松, 徐江玲, 艾波, 等. 基于SOA架构的国家海上搜救环境服务保障平台研发与应用[J]. 海洋预报, 2019, 36 (3): 71- 77.
	GAO S, XU J L, AI B, et al. National maritime search and rescue support platform based on service-oriented architecture[J]. Marine Forecasts, 2019, 36 (3): 71- 77.
27	AI B, LI B, GAO S, et al. An intelligent decision algorithm for the generation of maritime search and rescue emergency response plans[J]. IEEE Access, 2019, 7, 155835- 155850. doi: 10.1109/ACCESS.2019.2949366
28	FROST J R. Principles of search theory, part I: detection[J]. Response, 1999, 17 (2): 1- 7.
29	刘广强. 海上搜寻中确定扫海宽度的研究[D]. 大连: 大连海事大学, 2009.
	LIU G Q. Research on determining sweep widths for sea searches[D]. Dalian: Dalian Maritime University, 2009.
30	MELKMAN A A. On-line construction of the convex hull of a simple polyline[J]. Information Processing Letters, 1987, 25 (1): 11- 12. doi: 10.1016/0020-0190(87)90086-X
31	TOUSSAINT G T. Solving geometric problems with the rotating calipers[C]//Proc. of the IEEE Mediterranean Electrotechnical Conference, 1983, 83: 10−17.
32	MINSKY M. Steps toward artificial intelligence[J]. Proceedings of the IRE, 1963, 49 (1): 8- 30.

天气	搜索目标
天气	人员	救生筏
无风	1.0	1.0
风速大于28 km/h或浪高大于1 m	0.5	0.9
风速大于46 km/h或浪高大于1.5 m	0.25	0.6

对比内容	K-Means	DBSCAN
搜索单元数量	有限	充足
散点分布情况	均匀分布	集中分布
适用场景	已知搜索单元的数量，需进一步分配子区域	可根据聚类效果确定参加任务的搜索单元的数量
优势	可囊括全部散点	可去除部分噪声

参数	A	B	C
预测时间	2023.04.11 16:00	2023.03.30 16:00	2023.04.29 16:00
预测时长/h	12	12	48
预测起始坐标	114˚46'E 21˚00'N	114˚24'E 21˚08'N	116˚02'E 21˚17'N
搜索时段	9~10	3~5	2~4
风场参数	0.02	0.02	0.02
流场参数	1	1	1
风速/（km/h）	12.02	15.48	1.116
流速/（km/h）	1.00	1.30	4.08
能见度/n mile	17.05	15.25	9.68
搜索单元类型	商船	商船	航空器
搜索单元数量	1	2	4
有效扫视宽度/ n mile	0.7	0.7	0.1

[1]	Zhikang LIN, Jialei LIU, Jiazhi MA, Longfei SHI, Jinbao XU. Counter-anti-radiation method method using distributed radiation source blinking decoy [J]. Systems Engineering and Electronics, 2026, 48(1): 1-11.
[2]	Zeyu LU, Yao WANG, Weinan WU, Yiming SUN, Chunlin GONG. Two-layered path planning for unmanned aerial vehicle swarm in three-dimensional dynamic environment [J]. Systems Engineering and Electronics, 2026, 48(1): 132-143.
[3]	Jinyan XUE, Yasheng ZHANG, Xuefeng TAO, Mingqi YANG, Shuailong ZHAO. Advances in orbital maneuver control for GEO spacecraft [J]. Systems Engineering and Electronics, 2026, 48(1): 290-300.
[4]	Chuanlong SONG, Qianwu ZHANG, Jian HE, Wenjun ZHOU, Hui WANG, Weiwei KONG, Wenbo TIAN. Edge computing task offloading method of satellite-ground collaborative based on MADDPG algorithm [J]. Systems Engineering and Electronics, 2026, 48(1): 350-360.
[5]	Peng YAO, Meiyu HAN, Dechuan WANG, Zhicheng GAO. Multiple unmanned surface vehicles pursuit method based on adversarial evolutionary reinforcement learning [J]. Systems Engineering and Electronics, 2025, 47(9): 2960-2970.
[6]	Xiaolong WEI, Yarong WU, Dengkai YAO, Guhao ZHAO. Hierarchical decision-making algorithm for UAV air combat maneuvering based on deep reinforcement learning [J]. Systems Engineering and Electronics, 2025, 47(9): 2993-3003.
[7]	Dapeng YANG, Zihao GONG, Xiaoye WANG, Zhengyu GUO, Delin LUO. Research on UAV cooperative interception maneuver decision-making based on multi-agent reinforcement learning [J]. Systems Engineering and Electronics, 2025, 47(9): 3076-3085.
[8]	Zhao YANG, Jinbiao HU, Yan WANG, Hongbiao QI. UAV coverage path planning for mountain patrol considering different takeoff and landing nests [J]. Systems Engineering and Electronics, 2025, 47(8): 2622-2631.
[9]	Xiaowei FU, Xinyi WANG, Zhe QIAO. Attack-defense confrontation strategy of multi-UAV based on APIQ algorithm [J]. Systems Engineering and Electronics, 2025, 47(7): 2205-2215.
[10]	Jiahao LIU, Renjie XU, Maotong SUN, Jiuyao JIANG, Jichao LI, Kewei YANG. Reinforcement learning-based resilience optimization method of equipment system-of-systems [J]. Systems Engineering and Electronics, 2025, 47(7): 2216-2223.
[11]	Yundou ZHU, Haiquan SUN, Xiaoxuan HU. Multi-satellite cooperative imaging task planning method based on pointer network architecture [J]. Systems Engineering and Electronics, 2025, 47(7): 2246-2255.
[12]	Xiaozhen YAN, Xinyue ZHOU, Qinghua LUO. Improved A-star algorithm for dynamic path planning of unmanned ships [J]. Systems Engineering and Electronics, 2025, 47(7): 2314-2328.
[13]	Junchao TANG, Chunhe HU. Complete coverage path planning for UAVs in 3D terrain and wind field environment [J]. Systems Engineering and Electronics, 2025, 47(7): 2349-2356.
[14]	Xiaowei FU, Xinyi WANG, Zhe QIAO. Confront strategy of multi-unmanned aerial vehicle based on ASDDPG algorithm [J]. Systems Engineering and Electronics, 2025, 47(6): 1867-1879.
[15]	Linzhi MENG, Xiaojuan SUN, Yuxin HU, Bin GAO, Guoqing SUN, Wenhao MU. Reinforcement learning task scheduling algorithm for satellite on-orbit processing [J]. Systems Engineering and Electronics, 2025, 47(6): 1917-1929.

Path planning for maritime moving targets search based on reinforcement learning

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 20

References 32

Related Articles 15

Recommended Articles

Metrics

Comments