系统工程与电子技术 ›› 2026, Vol. 48 ›› Issue (7): 2469-2478.doi: 10.12305/j.issn.1001-506X.2026.07.30

• 通信与网络 • 上一篇    

基于分层强化学习的水下传感器网络路由协议

孙骞1, 范云霞1(), 张开越1, 叶方1, 李一兵1, 吕重阳2   

  1. 1. 哈尔滨工程大学信息与通信工程学院,黑龙江 哈尔滨 150001
    2. 哈尔滨理工大学理学院,黑龙江 哈尔滨 150001
  • 收稿日期:2025-05-28 修回日期:2025-11-04 出版日期:2026-01-24 发布日期:2026-01-24
  • 通讯作者: 孙骞 E-mail:2023994727@qq.com
  • 基金资助:
    黑龙江省优秀青年基金(YQ2024F017);国家自然科学基金(52271311);中央高校基本科研业务费(3072024XX0802);航空科学基金(2024Z024182001)资助课题

Hierarchical reinforcement learning-based routing protocol for underwater sensor networks

Qian SUN1, Yunxia FAN1(), Kaiyue ZHANG1, Fang YE1, Yibing LI1, Chongyang LYU2   

  1. 1. College of Information and Communication Engineering,Harbin Engineering University,Harbin 150001,China
    2. College of Science,Harbin University of Science and Technology,Harbin 150001,China
  • Received:2025-05-28 Revised:2025-11-04 Online:2026-01-24 Published:2026-01-24
  • Contact: Qian SUN E-mail:2023994727@qq.com

摘要:

水下传感器网络在数据传输过程中面临着高时延、高能耗等问题,尤其是在多跳传输场景下,时延和能耗问题更加突出。基于此,提出一种基于分层强化学习的水下路由协议,构建了高层和低层两级决策框架,通过可变权重的奖励函数,实现了时空自适应的多目标优化模型,引入空洞修复机制,智能化的状态监控和空洞检测机制确保数据能够绕过空洞区域,保持网络的连通性。仿真结果表明,基于动态分层Q学习的水下路由在面对不同节点密度、多源和动态拓扑的水下环境时,依然能够保持稳定、可靠和高效的性能,其数据包投递率更高、平均端到端时延更低、数据包能耗也显著优于传统方法,从而展现了在深海长周期监测中的应用潜力。

关键词: 水下传感器网络, 路由协议, 空洞修复

Abstract:

Underwater sensor networks data transmission faces significant challenges such as high latency and high energy consumption, particularly in multi-hop transmission scenarios where time delay and energy consumption issues become more pronounced. Based on this, a hierarchical reinforcement learning-based underwater routing protocol is proposed, which constructs a two-level decision-making framework comprising high-level and low-level decision layers. By implementing a reward function with variable weights, a spatiotemporal adaptive multi-objective optimization model is achieved. Additionally, a void repair mechanism is introduced, incorporating intelligent state monitoring and void detection to ensure that data can bypass void regions, thereby maintaining network connectivity. Simulation results demonstrate that dynamic Q-learning-based layered acoustic routing (DQLAR) can maintain stable, reliable, and efficient performance in underwater environments with varying node densities, multiple data sources, and dynamic topologies. Dynamic Q-learning-based layered acoustic routing exhibits higher packet delivery rates, lower average end-to-end latency, and significantly reduced energy consumption compared to conventional methods, highlighting its potential for deep-sea long-term monitoring applications.

Key words: underwater sensor networks, routing protocol, void repair

中图分类号: