基于深度强化学习的卫星光网络波长路由算法

doi:10.12305/j.issn.1001-506X.2023.01.31

Abstract

Abstract:

A method for optical satellite network wavelength routing which is based on deep reinforcement learning is proposed, aiming at the problems of slow route convergence and low wavelength utilization caused by the dynamic changes network topology, business diversification, and uneven load. Based on the software-defined (medium earth orbit/low earth orbit, MEO/LEO) two-layer satellite network architecture, the deep reinforcement learning algorithm is used to dynamically perceive the current network traffic load and link status, and a reward function based on delay, wavelength utilization and packet loss rate is constructed to make routing decisions. In order to solve the impact of a single-hop link on the entire optical path, a link bottleneck factor is introduced to search for the optimal path that meets the quality of service (QoS) constraints. The research results show that compared with the traditional satellite network distributed routing algorithm (SDRA) algorithm and the Q-routing algorithm, the proposed algorithm reduces the network delay and packet loss rate, improves the wavelength utilization, and also reduces the blocking rate of high-priority services.

Key words: optical satellite network, wavelength routing, deep reinforce learning, service quality

CLC Number:

TN915

Xin LI, Yongjun LI, Shanghong ZHAO. A wavelength routing algorithm for optical satellite network based on deep reinforcement learning[J]. Systems Engineering and Electronics, 2023, 45(1): 264-270.

Figures/Tables 11

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

References 30

1	LIU X F , YANG L , CHEN Q , et al. An analytic method of wavelength requirements in dynamic optical satellite networks[J]. IEEE Communications Letters, 2020, 24 (11): 2569- 2573. doi: 10.1109/LCOMM.2020.3009211
2	DONG Q R , CHEN T , GAO S J , et al. Progress of research on satellite-borne laser communication technology[J]. Chinese Optics, 2019, 12 (6): 1260- 1270. doi: 10.3788/co.20191206.1260
3	CHEN S Z , SUN S H , KANG S L . System integration of terrestrial mobile communication and satellite communication—the trends, challenges and key technologies in B5G and 6G[J]. China Communications, 2020, 17 (12): 156- 171. doi: 10.23919/JCC.2020.12.011
4	YANG Y , ZHU L D , MAO X , et al. The spread spectrum GFDM schemes for integrated satellite-terrestrial communication system[J]. China Communications, 2019, 16 (12): 165- 175. doi: 10.23919/JCC.2019.12.013
5	LI Y , LI K X , WANG J H , et al. Massive MIMO transmission for LEO satellite communications[J]. IEEE Journal on Selected Areas in Communications, 2020, 38 (8): 1851- 1865. doi: 10.1109/JSAC.2020.3000803
6	KUANG Q Q , ZHAN L , GU Z C , et al. High-energy passively mode-locked raman fiber laser pumped by a CW multimode laser[J]. Journal of Lightwave Technology, 2015, 33 (2): 391- 395. doi: 10.1109/JLT.2014.2375339
7	SUN Z X , XIAO R L , SU Z R , et al. High single-mode stability tunable in-series laser array with high wavelength-spacing uniformity[J]. Journal of Lightwave Technology, 2020, 38 (21): 6038- 6046. doi: 10.1109/JLT.2020.3005828
8	ERDOGAN E , ALTUNBAS I , KURT G K , et al. Site diversity in downlink optical satellite networks through ground station selection[J]. IEEE Access, 2021, 9, 31179- 31190. doi: 10.1109/ACCESS.2021.3059641
9	ZHENG Z L , HUA N , ZHONG Z Z , et al. Time-sliced flexible resource allocation for optical low earth orbit satellite networks[J]. IEEE Access, 2019, 7, 56753- 56759. doi: 10.1109/ACCESS.2019.2913441
10	LYRA N K , EFREM C N , KOUROGIORGAS C I , et al. Optimizing the ground network of optical MEO satellite communication systems[J]. IEEE Systems Journal, 2020, 14 (3): 3968- 3976. doi: 10.1109/JSYST.2019.2945838
11	GONG S L , SHEN H , ZHAO K L , et al. Network availability maximization for free-space optical satellite communications[J]. IEEE Wireless Communications Letters, 2020, 9 (3): 411- 415. doi: 10.1109/LWC.2019.2958804
12	刘庆利, 姚俊飞, 刘治国. 基于多业务的卫星光网络波长路由算法研究[J]. 系统仿真学报, 2017, 29 (8): 1780- 1787. doi: 10.16182/j.issn1004731x.joss.201708018
	LIU Q L , YAO J F , LIU Z G . Research on wavelength routing algorithm based on multi-service in satellite optical network[J]. Journal of System Simulation, 2017, 29 (8): 1780- 1787. doi: 10.16182/j.issn1004731x.joss.201708018
13	SUN X , CAO S Z . A routing and wavelength assignment algorithm based on two types of LEO constellations in optical satellite networks[J]. Journal of Lightwave Technology, 2020, 38 (8): 2106- 2113. doi: 10.1109/JLT.2020.2965185
14	YANG Q L , TAN L Y , MA J . Analysis of crosstalk in optical satellite networks with wavelength division multiplexing architectures[J]. Journal of Lightwave Technology, 2010, 28 (6): 931- 938. doi: 10.1109/JLT.2009.2035527
15	ZHOU R , HE L W , TANG C C , et al. SDN path enhancement algorithm based on multi-objective constraint genetic algoithm[J]. Computer Technology and Development, 2019, 7, 17- 22.
16	GUO Q , GU R T , DONG T , et al. SDN-based end-to-end fragment-aware routing for elastic data flows in LEO satellite-terrestrial network[J]. IEEE Access, 2019, 7, 396- 410. doi: 10.1109/ACCESS.2018.2885473
17	吴艳. 基于流量分类的智能SDN路由优化技术研究[D]. 杭州: 浙江工商大学, 2019.
	WU Y. Research on intelligent SDN routing optimization technology based on traffic classification[D]. Hangzhou: Zhejiang Gongshang University, 2019.
18	石晓东, 李勇军, 赵尚弘, 等. 基于SDN的卫星网络多QoS目标优化路由算法[J]. 系统工程与电子技术, 2020, 42 (6): 1395- 1401.
	SHI X D , LI Y J , ZHAO S H , et al. Multi-QoS objective optimization routing algorithm of satellite network based on SDN[J]. Systems Engineering and Electronics, 2020, 42 (6): 1395- 1401.
19	CHENG Z J , ZHANG X N , LI Y C , et al. Congestion-aware local reroute for fast failure recovery in software-defined networks[J]. Journal of Optical Communications and Networking, 2017, 9 (11): 934- 944. doi: 10.1364/JOCN.9.000934
20	PICCHI R , CHITI F , FANTACCI R , et al. Towards quantum satellite internetworking: a software-defined networking perspective[J]. IEEE Access, 2020, 8, 210370- 210381. doi: 10.1109/ACCESS.2020.3038529
21	LI T X , ZHOU H C , LUO H B , et al. SERvICE: a software defined framework for integrated space-terrestrial satellite communication[J]. IEEE Trans.on Mobile Computing, 2018, 17 (3): 703- 716. doi: 10.1109/TMC.2017.2732343
22	石晓东, 李勇军, 赵尚弘, 等. 软件定义卫星光网络蚁群优化波长路由技术[J]. 红外与激光工程, 2020, 49 (10): 211- 218.
	SHI X D , LI Y J , ZHAO S H , et al. Ant colony optimization routing and wavelength technology for software-defined satellite optical network[J]. Infrared and Laser Engineering, 2020, 49 (10): 211- 218.
23	STAMPA G, ARIAS M, SANCHEZ-CHARLES D, et al. A deep-reinforcement learning approach for software-defined networking routing optimization[J]. 2017, http://arxiv.org/abs/1709.0708.
24	YU C H , LAN J L , GUO Z H , et al. DROM: optimizing the routing in software-defined networks with deep reinforcement learning[J]. IEEE Access, 2018, 6, 64533- 64539. doi: 10.1109/ACCESS.2018.2877686
25	BOYAN J A , LITTMAN M L . Packet routing in dynamically changing networks: a reinforcement learning approach[J]. Advances Neural Information Processing Systems, 1994, 6
26	靳晓晗. 基于深度强化学习的无线多跳网络路由算法研究[D]. 北京: 中国科学院大学, 2020.
	JIN X H. Research on deep reinforcement learning based routing algorithm for wireless multihop networks[D]. Beijing: University of Chinese Academy of Sciences, 2020.
27	魏德宾, 刘健, 潘成胜, 等. 卫星网络中基于多QoS约束的蚁群优化路由算法[J]. 计算机工程, 2019, 45 (7): 114- 120.
	WEI D B , LIU J , PAN C S , et al. Ant colony optimization routing algorithm based on multi-QoS constraints in satellite networks[J]. Computer Engineering, 2019, 45 (7): 114- 120.
28	QIU C , YAO H , YU F R , et al. Deep Q-learning aided networking, caching, and computing resources allocation in software-defined satellite-terrestrial networks[J]. IEEE Trans.on Vehicular Technology, 2019, 68 (6): 5871- 5883.
29	GUO A D, ZHAO C L, XU F M, et al. LEO satellite routing algorithm in software defined space terrestrial integrated network[C]// Proc. of the International Symposium on Communications and Information Technologies, 2017: 1-6.
30	KIM S, SON J, TALUKDER A, et al. Congestion prevention mechanism based on Q-learning for efficient routing in SDN[C]// Proc. of the International Conference on Information Networking, 2016.

参数	数值
LEO卫星数量	66
LEO轨道数量	6
LEO轨内卫星数	11
高度/km	780
LEO运行周期/min	114
轨内星间链路数	60
轨间星间链路数	55
倾斜角/(°)	86.4
星间链路距离/km	4 033
星间链路波长数	6

[1]	Xinkang SONG, Shanghong ZHAO, Xiang WANG, Shaowei HAO. Collaborative construction and embedding strategy of aviation information network service function chain [J]. Systems Engineering and Electronics, 2022, 44(11): 3556-3563.
[2]	Xiaodong SHI, Yongjun LI, Shanghong ZHAO, Weilong WANG, Xingyu WANG. Multi-QoS objective optimization routing algorithm of satellite network based on SDN [J]. Systems Engineering and Electronics, 2020, 42(6): 1395-1401.
[3]	ZHENG Wei, LIU San-yang, KOU Xiao-li. Energy efficient routing algorithm based on ant colony optimization for wireless sensor network [J]. Journal of Systems Engineering and Electronics, 2009, 31(8): 1993-1996.

A wavelength routing algorithm for optical satellite network based on deep reinforcement learning

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 30

Related Articles 3

Recommended Articles

Metrics

Comments