系统工程与电子技术 ›› 2023, Vol. 45 ›› Issue (1): 264-270.doi: 10.12305/j.issn.1001-506X.2023.01.31

• 通信与网络 • 上一篇    

基于深度强化学习的卫星光网络波长路由算法

李信, 李勇军, 赵尚弘   

  1. 空军工程大学信息与导航学院, 陕西 西安 710003
  • 收稿日期:2021-07-04 出版日期:2023-01-01 发布日期:2023-01-03
  • 通讯作者: 李信
  • 作者简介:李信(1997—), 男, 博士研究生, 主要研究方向为卫星光网络波长路由算法
    李勇军(1979—), 男, 副教授, 博士研究生导师, 博士, 主要研究方向为卫星光通信与网络
    赵尚弘(1964—), 男, 教授, 博士研究生导师, 博士, 主要研究方向为激光原理技术
  • 基金资助:
    国家自然科学基金(91638101);国家自然科学基金(61701522);陕西省自然科学基金(2018JM6069)

A wavelength routing algorithm for optical satellite network based on deep reinforcement learning

Xin LI, Yongjun LI, Shanghong ZHAO   

  1. Information and Navigation College, Air Force Engineering University, Xi'an 710003, China
  • Received:2021-07-04 Online:2023-01-01 Published:2023-01-03
  • Contact: Xin LI

摘要:

针对由卫星光网络拓扑动态变化、业务多样化和负载不均引起的路由收敛慢和波长利用率低的问题, 提出了一种基于深度强化学习的卫星光网络波长路由分配方法。基于软件定义中轨/低轨(medium earth orbit/low earth orbit, MEO/LEO)双层卫星网络架构, 利用深度强化学习算法动态感知网络当前的业务负载和链路状况, 构造基于时延、波长利用率和丢包率的奖励函数进行选路决策。为了解决单跳链路对整个光路的影响, 引入链路瓶颈因子, 搜索符合服务质量(quality of service, QoS)约束的最优路径。研究结果表明, 与传统卫星网络分布式路由(satellite network distributed routing algorithm, SDRA)算法和Q-routing算法相比, 所提算法降低了网络的时延、丢包率, 提高了波长利用率, 同时也降低了高优先级业务的阻塞率。

关键词: 卫星光网络, 波长路由, 深度强化学习, 服务质量

Abstract:

A method for optical satellite network wavelength routing which is based on deep reinforcement learning is proposed, aiming at the problems of slow route convergence and low wavelength utilization caused by the dynamic changes network topology, business diversification, and uneven load. Based on the software-defined (medium earth orbit/low earth orbit, MEO/LEO) two-layer satellite network architecture, the deep reinforcement learning algorithm is used to dynamically perceive the current network traffic load and link status, and a reward function based on delay, wavelength utilization and packet loss rate is constructed to make routing decisions. In order to solve the impact of a single-hop link on the entire optical path, a link bottleneck factor is introduced to search for the optimal path that meets the quality of service (QoS) constraints. The research results show that compared with the traditional satellite network distributed routing algorithm (SDRA) algorithm and the Q-routing algorithm, the proposed algorithm reduces the network delay and packet loss rate, improves the wavelength utilization, and also reduces the blocking rate of high-priority services.

Key words: optical satellite network, wavelength routing, deep reinforce learning, service quality

中图分类号: