弱通信下无人潜航器事件触发一致性协同控制

doi:10.12305/j.issn.1001-506X.2025.02.25

摘要/Abstract

摘要：

针对水下无人潜航器(unmanned underwater vehicle, UUV)集群在弱通信条件下的一致性协同控制问题, 考虑水下群间通信存在的高延时、低带宽、需具有隐蔽性等弱通信特点, 设计基于强化学习的事件触发智能一致性协同控制架构, 以实现UUV集群在弱通信条件下的有效协同。首先, 设计一个事件触发分布式观测器, 该观测器利用领导者与邻居的动态交互信息, 来估计弱通信条件下UUV所需的跟踪参考信号。随后, 采用强化学习方法直接从系统交互中学习最优控制策略。最后, 通过仿真结果验证了所提方法的有效性。

关键词: 弱通信, 事件触发, 一致性协同控制, 强化学习

Abstract:

Aiming at the problem of consensus collaborative control of unmanned underwater vehicle (UUV) clusters under adverse communication conditions, considering the adverse communication characteristics of high latency, low bandwidth, and the need for stealth in underwater inter-cluster communication, a reinforcement learning based event-triggered intelligent consensus collaborative control architecture is designed to achieve effective collaboration of UUV clusters under adverse communication conditions. Firstly, an event-triggered distributed observer is devised which utilizes dynamic interaction information between leaders and neighbors to estimate the tracking reference signal required for UUVs under adverse communication conditions. Subsequently, reinforcement learning methods are used to directly learn the optimal control strategy from system interactions. Finally, the effectiveness of the proposed method is validated through simulation results.

Key words: adverse communication condition, event-triggered, consensus cooperative control, reinforce-ment learning

中图分类号:

TP13

赵万兵, 夏元清, 戴荔, 张元. 弱通信下无人潜航器事件触发一致性协同控制[J]. 系统工程与电子技术, 2025, 47(2): 591-599.

Wanbing ZHAO, Yuanqing XIA, Li DAI, Yuan ZHANG. Event-triggered consensus cooperative control of unmanned underwater vehicle under adverse communication condition[J]. Systems Engineering and Electronics, 2025, 47(2): 591-599.

图/表 8

图1

图2

图3

图4

图5

图6

图7

图8

参考文献 33

9	YANG H Z , WANG C F , ZHANG F M . A decoupled controller design approach for formation control of autonomous underwater vehicles with time delays[J]. IET Control Theory & Applications, 2013, 7 (15): 1950- 1958.
10	YAN Z P , YANG Z W , YUE L D , et al. Discrete-time coordinated control of leader-following multiple AUVs under switch-ing topologies and communication delays[J]. Ocean Engineering, 2019, 172, 361- 372.
11	SURYENDU C , SUBUDHI B . Formation control of multiple autonomous underwater vehicles under communication delays[J]. IEEE Trans.on Circuits and Systems Ⅱ: Express Briefs, 2020, 67 (12): 3182- 3186.
12	LIU K X , LU J H , LIN Z L . Design of distributed observers in the presence of arbitrarily large communication delays[J]. IEEE Trans.on Neural Networks and Learning Systems, 2018, 29 (9): 4447- 4461.
13	SU B , WANG H B , WANG Y L . Dynamic event-triggered formation control for AUVs with fixed-time integral sliding mode distur-bance observer[J]. Ocean Engineering, 2021, 240, 109893.
14	SHI Y , XIE W , ZHANG G Q , et al. Event-triggered saturation-tolerant control for autonomous underwater vehicles with quantitative transient behaviors[J]. IEEE Trans.on Vehicular Technology, 2023, 72 (8): 9857- 9867.
15	XU Y Y , LI T S , TONG S C . Event-triggered adaptive fuzzy bipartite consensus control of multiple autonomous underwater vehicles[J]. IET Control Theory & Applications, 2020, 14 (20): 3632- 3642.
16	MENG C C , ZHANG W , DU X . Finite-time extended state observer based collision-free leaderless formation control of multiple AUVs via event-triggered control[J]. Ocean Engineering, 2023, 268, 113605.
17	YAN Z P , ZHANG C , ZHANG M Y , et al. Distributed event-triggered formation control for multi-AUV system via asynchronous periodic sampling control approach[J]. Ocean Engineering, 2022, 256, 111561.
18	许雅筑, 武辉, 游科友, 等. 强化学习方法在自主水下机器人控制任务中的应用[J]. 中国科学(信息科学), 2020, 50 (12): 1798- 1816.
	XU Y Z , WU H , YOU K Y , et al. A selected review of reinforcement learning-based control for autonomous underwater vehicles[J]. Science in China (Information Sciences), 2020, 50 (12): 1798- 1816.
19	WU H , SONG S J , YOU K Y , et al. Depth control of model-free AUVs via reinforcement learning[J]. IEEE Trans.on Systems, Man, and Cybernetics: Systems, 2019, 49 (12): 2499- 2510.
20	CUI R X , YANG C G , LI Y , et al. Adaptive neural network control of AUVs with control input nonlinearities using reinforcement learning[J]. IEEE Trans.on Systems, Man, and Cybernetics: Systems, 2017, 47 (6): 1019- 1029.
21	CAO W Q , YAN J , YANG X , et al. Communication-aware formation control of AUVs with model uncertainty and fading channel via integral reinforcement learning[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10 (1): 159- 176.
22	WANG Z K , ZHANG L J . Distributed optimal formation tracking control based on reinforcement learning for underactuated AUVs with asymmetric constraints[J]. Ocean Engineering, 2023, 280, 114491.
23	ZHAO W B , XIA Y Q , ZHAI D H , et al. Adaptive event-tri-ggered coordination control of unknown autonomous underwater vehicles under communication link faults[J]. Automatica, 2023, 158, 111277.
24	HOU S P , CHEAH C C . Can a simple control scheme work for a formation control of multiple autonomous underwater vehicles?[J]. IEEE Trans.on Control Systems Technology, 2011, 19 (5): 1090- 1101.
25	LIU H , WANG Y H , LEWIS F L . Robust distributed formation controller design for a group of unmanned underwater vehicles[J]. IEEE Trans.on Systems, Man, and Cybernetics: Systems, 2021, 51 (2): 1215- 1223.
26	CAI H , HUANG J . The leader-following consensus for multiple uncertain euler-lagrange systems with an adaptive distributed observer[J]. IEEE Trans.on Automatic Control, 2016, 61 (10): 3152- 3157.
27	MU B X , ZHANG K W , SHI Y . Integral sliding mode flight controller design for a quadrotor and the application in a heterogeneous multi-agent system[J]. IEEE Trans.on Industrial Electronics, 2017, 64 (12): 9389- 9398.
28	HU W F , LIU L , FENG G . Cooperative output regulation of linear multi-agent systems by intermittent communication: a unified framework of time and event-triggering strategies[J]. IEEE Trans.on Automatic Control, 2018, 63 (2): 548- 555.
1	闫敬, 关新平, 罗小元, 等. 水下信息物理系统探测-通信-控制一体化: 挑战与进展[J]. 控制理论与应用, 2022, 39 (11): 1996- 2008.
	YAN J , GUAN X P , LUO X Y , et al. Integration of detection, communication, and control in underwater information physical systems: challenges and progress[J]. Control Theory and App-lications, 2022, 39 (11): 1996- 2008.
2	曾斌, 张鸿强, 李厚朴. 针对无人潜航器的反潜策略研究[J]. 系统工程与电子技术, 2022, 44 (10): 3174- 3181. doi: 10.12305/j.issn.1001-506X.2022.10.21
	ZENG B , ZHANG H Q , LI H P . Research on anti-submarine strategy for unmanned underwater vehicles[J]. Systems Engineering and Electronics, 2022, 44 (10): 3174- 3181. doi: 10.12305/j.issn.1001-506X.2022.10.21
3	韩东, 贺寅, 陈立军, 等. 水下通信技术及其难点[J]. 科技创新与应用, 2021 (1): 155- 159.
	HAN D , HE Y , CHEN L J , et al. Underwater communication technology and its challenges[J]. Science and Technology Innovation and Application, 2021 (1): 155- 159.
4	胡桥, 赵振轶, 冯豪博, 等. AUV智能集群协同任务研究进展[J]. 水下无人系统学报, 2023, 31 (2): 189- 200.
	HU Q , ZHAO Z Y , FENG H B , et al. Research progress on collaborative missions of AUV intelligent swarms[J]. Journal of Underwater Unmanned Systems, 2023, 31 (2): 189- 200.
5	LIANG H T , CAO H , FU Y F . Decentralized adaptive flocking control algorithm with avoiding collision and preserving connectivity for crowded UUV swarm with uncertainties and input satu-ration[J]. Ocean Engineering, 2021, 237, 109545. doi: 10.1016/j.oceaneng.2021.109545
6	WEI W , WANG J J , FANG Z R , et al. 3U: joint design of UAV-USV-UUV networks for cooperative target hunting[J]. IEEE Trans.on Vehicular Technology, 2023, 72 (3): 4085- 4090. doi: 10.1109/TVT.2022.3220856
7	LI H P , XIE P , YAN W S . Receding horizon formation tracking control of constrained underactuated autonomous underwater vehi-cles[J]. IEEE Trans.on Industrial Electronics, 2017, 64 (6): 5004- 5013. doi: 10.1109/TIE.2016.2589921
8	LI X , ZHU D Q . An adaptive SOM neural network method for distributed formation control of a group of AUVs[J]. IEEE Trans.on Industrial Electronics, 2018, 65 (10): 8260- 8270.
29	ZHANG W , ZENG J , YAN Z P , et al. Leader-following consensus of discrete-time multi-AUV recovery system with time-varying delay[J]. Ocean Engineering, 2021, 219, 108258.
30	QIAN Y Y , LIU L , FENG G . Distributed dynamic event-triggered control for cooperative output regulation of linear multi-agent systems[J]. IEEE Trans.on Cybernetics, 2020, 50 (7): 3023- 3032.
31	MODARES H , LEWIS F L , NAGHIBI-SISTANI M . Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems[J]. Automatica, 2014, 50 (1): 193- 202.
32	ZHAO W B , LIU H , LEWIS F L . Robust formation control for cooperative underactuated quadrotors via reinforcement learning[J]. IEEE Trans.on Neural Networks and Learning Systems, 2021, 32 (10): 4577- 4587.
33	MODARES H , LEWIS F L , KANG W , et al. Optimal synchronization of heterogeneous nonlinear systems with unknown dynamics[J]. IEEE Trans.on Automatic Control, 2018, 63 (1): 117- 131.

[1]	付可, 陈浩, 王宇, 刘权, 黄健. 基于不确定性的贝叶斯策略重用方法[J]. 系统工程与电子技术, 2025, 47(2): 535-543.
[2]	刘晓琳, 郭梦娇, 李卓. Dueling DQN优化下的航班延误自适应图卷积循环网络预测方法[J]. 系统工程与电子技术, 2025, 47(2): 568-579.
[3]	闫循良, 王宽, 张子剑, 王培臣. 基于LSTM-DDPG的再入制导方法[J]. 系统工程与电子技术, 2025, 47(1): 268-279.
[4]	张庭瑜, 曾颖, 李楠, 黄洪钟. 基于深度强化学习的航天器功率-信号复合网络优化算法[J]. 系统工程与电子技术, 2024, 46(9): 3060-3069.
[5]	夏雨奇, 黄炎焱, 陈恰. 基于深度Q网络的无人车侦察路径规划[J]. 系统工程与电子技术, 2024, 46(9): 3070-3081.
[6]	杨志鹏, 陈子浩, 曾长, 林松, 毛金娣, 张凯. 复杂环境下的飞行器在线航路规划决策方法[J]. 系统工程与电子技术, 2024, 46(9): 3166-3175.
[7]	彭莉莎, 孙宇祥, 薛宇凡, 周献中. 融合三支多属性决策与SAC的兵棋推演智能决策技术[J]. 系统工程与电子技术, 2024, 46(7): 2310-2322.
[8]	郭宏达, 娄静涛, 徐友春, 叶鹏, 李永乐, 陈晋生. 基于MADDPG的多无人车协同事件触发通信[J]. 系统工程与电子技术, 2024, 46(7): 2525-2533.
[9]	张梦钰, 豆亚杰, 陈子夷, 姜江, 杨克巍, 葛冰峰. 深度强化学习及其在军事领域中的应用综述[J]. 系统工程与电子技术, 2024, 46(4): 1297-1308.
[10]	李彦铃, 罗飞舟, 葛致磊. 基于鲁棒观测器的深度强化学习垂直起降运载器姿态稳定研究[J]. 系统工程与电子技术, 2024, 46(3): 1038-1047.
[11]	禹志龙, 李颖晖, 裴彬彬, 段效聪, 张喆. 基于事件触发采样的无尾翼飞机自适应容错姿态控制[J]. 系统工程与电子技术, 2024, 46(3): 1058-1066.
[12]	唐恒, 孙伟, 吕磊, 贺若飞, 吴建军, 孙昌浩, 孙田野. 融合动态奖励策略的无人机编队路径规划方法[J]. 系统工程与电子技术, 2024, 46(10): 3506-3518.
[13]	冯路为, 刘松涛, 徐华志. 基于POMDP模型的智能雷达干扰决策方法[J]. 系统工程与电子技术, 2023, 45(9): 2755-2760.
[14]	马悦, 吴琳, 许霄. 基于多智能体强化学习的协同目标分配[J]. 系统工程与电子技术, 2023, 45(9): 2793-2801.
[15]	韦道知, 张曌宇, 谢家豪, 李宁. 基于改进Actor-Critic算法的多传感器交叉提示技术[J]. 系统工程与电子技术, 2023, 45(6): 1624-1632.