基于强化学习的海上移动目标搜索路径规划

doi:10.12305/j.issn.1001-506X.2026.02.13

系统工程与电子技术 ›› 2026, Vol. 48 ›› Issue (2): 515-523.doi: 10.12305/j.issn.1001-506X.2026.02.13

• 系统工程 • 上一篇

基于强化学习的海上移动目标搜索路径规划

杨鹏程¹(), 杨清清¹, 高盈盈¹^,*, 杨志伟¹, 杨克巍¹, 艾波²

1. 国防科技大学系统工程学院，湖南长沙 410073
2. 山东科技大学测绘与空间信息学院，山东青岛 266590

收稿日期:2023-09-28 修回日期:2024-03-06 出版日期:2024-06-14 发布日期:2024-06-14
通讯作者: 高盈盈 E-mail:15206889290@163.com
作者简介:杨鹏程（2001—），男，硕士研究生，主要研究方向为应急管理与智能决策
杨清清（1982—），女，教授，博士，主要研究方向为应急管理智能决策、资源优化与任务规划方法
杨志伟（1988—），男，副教授，博士，主要研究方向为装备体系分析评估与优化、智能优化算法
杨克巍（1977—），男，教授，博士，主要研究方向为大数据与体系工程、复杂系统机理
艾　波（1979—），男，教授，博士，主要研究方向为海洋地理信息系统
基金资助:
国家自然科学基金（72001209,72231011,72374209）；湖南省自然科学基金（2023JJ30641,2022JJ20047）资助课题

Path planning for maritime moving targets search based on reinforcement learning

Pengcheng YANG¹(), Qingqing YANG¹, Yingying GAO¹^,*, Zhiwei YANG¹, Kewei YANG¹, Bo AI²

1. College of Systems Engineering，National University of Defense Technology，Changsha 410073，China
2. College of Geodesy and Geomatics，Shandong University of Science and Technology，Qingdao 266590，China

Received:2023-09-28 Revised:2024-03-06 Online:2024-06-14 Published:2024-06-14
Contact: Yingying GAO E-mail:15206889290@163.com

摘要/Abstract

摘要：

海上移动目标搜索是海上搜救行动的重要环节，搜索效率直接影响搜救行动的成功率。针对海上移动目标搜索路径规划问题进行了以下三部分工作。首先，提出基于最小覆盖矩形的搜索区域确定算法，以划分最佳搜索区域；其次，构建基于强化学习的搜索路径规划算法，综合考虑多重约束条件，实现了子区域的最佳搜索路径规划；最后，研发决策支持系统，验证了算法的有效性和鲁棒性，并扩展到多搜索单元合作的场景。研究结果表明，所提算法在不同搜索场景下均有良好的求解效果，对提高海上搜救行动的成功率具有重要的理论意义和应用价值。

关键词: 海上移动目标搜索, 强化学习, 路径规划, 决策支持

Abstract:

Maritime moving target search plays a crucial role in maritime search and rescue operations, as the search efficiency directly impacts the success rate of the operations. Aiming at the maritime moving target search path planning problem, the following three parts of work are done. Firstly, an algorithm to determine the search area based on the minimum coverage rectangle is proposed to divide the optimal search area. Next, a search path planning algorithm with reinforcement learning is constructed, comprehensively considering multiple constraints, and achieving optimal search path planning of sub-areas. Finally, a decision support system is developed to verify the effectiveness and robustness of the algorithms and it is extended to scenarios involving multi-search unit cooperation. The results demonstrate that the proposed algorithms have achieved positive outcomes across a spectrum of search scenarios. This showcases not only considerable theoretical significance but also practical potential in enhancing the success rate of search and rescue operations.

Key words: maritime moving targets search, reinforcement learning, path planning, decision support

中图分类号:

N 945.15

杨鹏程, 杨清清, 高盈盈, 杨志伟, 杨克巍, 艾波. 基于强化学习的海上移动目标搜索路径规划[J]. 系统工程与电子技术, 2026, 48(2): 515-523.

Pengcheng YANG, Qingqing YANG, Yingying GAO, Zhiwei YANG, Kewei YANG, Bo AI. Path planning for maritime moving targets search based on reinforcement learning[J]. Systems Engineering and Electronics, 2026, 48(2): 515-523.

图/表 20

表1

图1

表2

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

表3

图14

图15

图16

图17

参考文献 32

1	ZHENG L X, WU M Q, ZHAO J, et al. Effects of Ulva prolifera dissipation on the offshore environment based on remote sensing images and field monitoring data[J]. Acta Oceanologica Sinica, 2023, 42 (6): 112- 120. doi: 10.1007/s13131-022-2129-7
2	YANG D, YUEN K V, GU X F, et al. Influences of environmental factors on the dissipation of green tides in the Yellow Sea, China[J]. Marine Pollution Bulletin, 2023, 189, 114737. doi: 10.1016/j.marpolbul.2023.114737
3	LIU Z K, HAN Z H, CHEN Q, et al. Risk assessment of marine oil spills using dynamic Bayesian network analyses[J]. Environmental Pollution, 2023, 317, 120716. doi: 10.1016/j.envpol.2022.120716
4	LI J Q, ZHANG G Q, JIANG C Y, et al. A survey of maritime unmanned search system: theory, applications and future directions[J]. Ocean Engineering, 2023, 285, 115359. doi: 10.1016/j.oceaneng.2023.115359
5	KOOPMAN B O. The theory of search. I. kinematic bases[J]. Operations Research, 1956, 4 (3): 324- 346. doi: 10.1287/opre.4.3.324
6	KOOPMAN B O. The theory of search. II. target detection[J]. Operations Research, 1956, 4 (5): 503- 531. doi: 10.1287/opre.4.5.503
7	KOOPMAN B O. The theory of search: III. the optimum distribution of searching effort[J]. Operations Research, 1957, 5 (5): 613- 626. doi: 10.1287/opre.5.5.613
8	杨克巍, 梁笑天, 郭玙, 等. 海上搜寻理论方法与搜救决策支持系统现状及发展[J]. 海岸工程, 2021, 40 (4): 291- 302. doi: 10.3969/j.issn.1002-3682.2021.04.006
	YANG K W, LIANG X T, GUO Y, et al. Status and development of maritime search theory and search and rescue decision support system[J]. Coastal Engineering, 2021, 40 (4): 291- 302. doi: 10.3969/j.issn.1002-3682.2021.04.006
9	BREIVIK O, ALLEN A A. An operational search and rescue model for the Norwegian Sea and the North Sea[J]. Journal of Marine Systems, 2008, 69 (1/2): 99- 113. doi: 10.1016/j.jmarsys.2007.02.010
10	AN D D, ZHANG T C, ZHANG J, et al. Research on drift path prediction technology of ships[C]//Proc. of the International Conference on Computer Modeling, Simulation and Algorithm, 2018: 231−235.
11	BURCIU Z. Application of Fokker-Planck equation for modeling the search and rescue area at sea[J]. Annual of Navigation, 2002, 4, 21- 32.
12	OTOTE D A, LI B, AI B, et al. A decision-making algorithm for maritime search and rescue plan[J]. Sustainability, 2019, 11 (7): 2084. doi: 10.3390/su11072084
13	ARKIN E M, FEKETE S P, MITCHELL J S B. Approximation algorithms for lawn mowing and milling[J]. Computational Geometry, 2000, 17 (1/2): 25- 50. doi: 10.1016/S0925-7721(00)00015-8
14	HERMANS B, LEUS R, MATUSCHKE J. Exact and approximation algorithms for the expanding search problem[J]. INFORMS Journal on Computing, 2022, 34 (1): 281- 296. doi: 10.1287/ijoc.2020.1047
15	MORIN M, ABI-ZEID I, QUIMPER C G. Ant colony optimization for path planning in search and rescue operations[J]. European Journal of Operational Research, 2023, 305 (1): 53- 63. doi: 10.1016/j.ejor.2022.06.019
16	LI X W, GAO M, KANG Z, et al. Collaborative search and rescue based on swarm of H-MASSs using consensus theory[J]. Ocean Engineering, 2023, 278, 114426. doi: 10.1016/j.oceaneng.2023.114426
17	DAH-ACHINANON U, BAJESTANI S E M, LAJOIE P Y, et al. Search and rescue with sparsely connected swarms[J]. Autonomous Robots, 2023, 47 (7): 849- 863. doi: 10.1007/s10514-022-10080-7
18	LAPORTE G, LOUVEAUX F, MERCURE H. Models and exact solutions for a class of stochastic location-routing problems[J]. European Journal of Operational Research, 1989, 39 (1): 71- 78. doi: 10.1016/0377-2217(89)90354-8
19	AGNETIS A, HERMANS B, LEUS R, et al. Time-critical testing and search problems[J]. European Journal of Operational Research, 2022, 296 (2): 440- 452. doi: 10.1016/j.ejor.2021.03.038
20	李苯帅. 基于最优搜寻理论的海上搜救方案规划方法研究[D]. 青岛: 山东科技大学, 2020.
	LI B S. Research on planning method of maritime search and rescue based on optimal search theory[D]. Qingdao: Shandong University of Science and Technology, 2020.
21	CHO S W, PARK H J, LEE H, et al. Coverage path planning for multiple unmanned aerial vehicles in maritime search and rescue operations[J]. Computers & Industrial Engineering, 2021, 161, 107612.
22	SUTTON R S, BARTO A G. Reinforcement learning: an introduction[M]. Cambridge: MIT Press, 2018.
23	AI B, JIA M X, XU H W, et al. Coverage path planning for maritime search and rescue using reinforcement learning[J]. Ocean Engineering, 2021, 241, 110098. doi: 10.1016/j.oceaneng.2021.110098
24	WOO J, YU C, KIM N. Deep reinforcement learning-based controller for path following of an unmanned surface vehicle[J]. Ocean Engineering, 2019, 183, 155- 166. doi: 10.1016/j.oceaneng.2019.04.099
25	杨清清, 高盈盈, 郭玙, 等. 基于深度强化学习的海战场目标搜寻路径规划[J]. 系统工程与电子技术, 2022, 44 (11): 3486- 3495. doi: 10.12305/j.issn.1001-506X.2022.11.24
	YANG Q Q, GAO Y Y, GUO Y, et al. Target search path planning for naval battle field based on deep reinforcement learning[J]. Systems Engineering and Electronics, 2022, 44 (11): 3486- 3495. doi: 10.12305/j.issn.1001-506X.2022.11.24
26	高松, 徐江玲, 艾波, 等. 基于SOA架构的国家海上搜救环境服务保障平台研发与应用[J]. 海洋预报, 2019, 36 (3): 71- 77.
	GAO S, XU J L, AI B, et al. National maritime search and rescue support platform based on service-oriented architecture[J]. Marine Forecasts, 2019, 36 (3): 71- 77.
27	AI B, LI B, GAO S, et al. An intelligent decision algorithm for the generation of maritime search and rescue emergency response plans[J]. IEEE Access, 2019, 7, 155835- 155850. doi: 10.1109/ACCESS.2019.2949366
28	FROST J R. Principles of search theory, part I: detection[J]. Response, 1999, 17 (2): 1- 7.
29	刘广强. 海上搜寻中确定扫海宽度的研究[D]. 大连: 大连海事大学, 2009.
	LIU G Q. Research on determining sweep widths for sea searches[D]. Dalian: Dalian Maritime University, 2009.
30	MELKMAN A A. On-line construction of the convex hull of a simple polyline[J]. Information Processing Letters, 1987, 25 (1): 11- 12. doi: 10.1016/0020-0190(87)90086-X
31	TOUSSAINT G T. Solving geometric problems with the rotating calipers[C]//Proc. of the IEEE Mediterranean Electrotechnical Conference, 1983, 83: 10−17.
32	MINSKY M. Steps toward artificial intelligence[J]. Proceedings of the IRE, 1963, 49 (1): 8- 30.

天气	搜索目标
天气	人员	救生筏
无风	1.0	1.0
风速大于28 km/h或浪高大于1 m	0.5	0.9
风速大于46 km/h或浪高大于1.5 m	0.25	0.6

对比内容	K-Means	DBSCAN
搜索单元数量	有限	充足
散点分布情况	均匀分布	集中分布
适用场景	已知搜索单元的数量，需进一步分配子区域	可根据聚类效果确定参加任务的搜索单元的数量
优势	可囊括全部散点	可去除部分噪声

参数	A	B	C
预测时间	2023.04.11 16:00	2023.03.30 16:00	2023.04.29 16:00
预测时长/h	12	12	48
预测起始坐标	114˚46'E 21˚00'N	114˚24'E 21˚08'N	116˚02'E 21˚17'N
搜索时段	9~10	3~5	2~4
风场参数	0.02	0.02	0.02
流场参数	1	1	1
风速/（km/h）	12.02	15.48	1.116
流速/（km/h）	1.00	1.30	4.08
能见度/n mile	17.05	15.25	9.68
搜索单元类型	商船	商船	航空器
搜索单元数量	1	2	4
有效扫视宽度/ n mile	0.7	0.7	0.1

[1]	林志康, 刘甲磊, 马佳智, 施龙飞, 徐进宝. 利用分布式辐射源闪烁诱偏的抗反辐射方法[J]. 系统工程与电子技术, 2026, 48(1): 1-11.
[2]	薛锦妍, 张雅声, 陶雪峰, 杨茗棋, 赵帅龙. GEO航天器轨道机动控制研究进展[J]. 系统工程与电子技术, 2026, 48(1): 290-300.
[3]	宋传龙, 张倩武, 何健, 周文骏, 王辉, 孔巍巍, 田文波. 基于MADDPG算法的星地协同边缘计算任务卸载方法[J]. 系统工程与电子技术, 2026, 48(1): 350-360.
[4]	姚鹏, 韩美玉, 王德川, 高志诚. 基于对抗进化强化学习的多无人艇追捕方法[J]. 系统工程与电子技术, 2025, 47(9): 2960-2970.
[5]	魏潇龙, 吴亚荣, 姚登凯, 赵顾颢. 基于深度强化学习的无人机空战机动分层决策算法[J]. 系统工程与电子技术, 2025, 47(9): 2993-3003.
[6]	杨大鹏, 龚资浩, 王小也, 郭正玉, 罗德林. 基于多智能体强化学习的无人机协同截击机动决策研究[J]. 系统工程与电子技术, 2025, 47(9): 3076-3085.
[7]	羊钊, 胡锦标, 王艳, 齐洪彪. 考虑异巢起降的无人机山地巡检覆盖路径规划[J]. 系统工程与电子技术, 2025, 47(8): 2622-2631.
[8]	符小卫, 王辛夷, 乔哲. 基于APIQ算法的多无人机攻防对抗策略[J]. 系统工程与电子技术, 2025, 47(7): 2205-2215.
[9]	柳佳豪, 徐任杰, 孙茂桐, 姜九瑶, 李际超, 杨克巍. 基于强化学习的装备体系韧性优化方法[J]. 系统工程与电子技术, 2025, 47(7): 2216-2223.
[10]	朱运豆, 孙海权, 胡笑旋. 基于指针网络架构的多星协同成像任务规划方法[J]. 系统工程与电子技术, 2025, 47(7): 2246-2255.
[11]	焉晓贞, 周新悦, 罗清华. 改进A-star算法的无人船动态路径规划[J]. 系统工程与电子技术, 2025, 47(7): 2314-2328.
[12]	唐俊超, 胡春鹤. 三维地形风场环境下无人机全覆盖路径规划[J]. 系统工程与电子技术, 2025, 47(7): 2349-2356.
[13]	符小卫, 王辛夷, 乔哲. 基于ASDDPG算法的多无人机对抗策略[J]. 系统工程与电子技术, 2025, 47(6): 1867-1879.
[14]	孟麟芝, 孙小涓, 胡玉新, 高斌, 孙国庆, 牟文浩. 面向卫星在轨处理的强化学习任务调度算法[J]. 系统工程与电子技术, 2025, 47(6): 1917-1929.
[15]	刘伊婕, 姜斌, 马亚杰, 李文博, 刘成瑞. 无人艇编队避碰路径规划与重规划[J]. 系统工程与电子技术, 2025, 47(6): 1964-1974.

基于强化学习的海上移动目标搜索路径规划

Path planning for maritime moving targets search based on reinforcement learning

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 20

参考文献 32

相关文章 15

编辑推荐

Metrics

本文评价