改进的低时延全回波Q路由算法

doi:10.3969/j.issn.1001-506X.2020.04.27

Abstract

Abstract:

Aiming at the congestion problem caused by traditional routing algorithms which cannot adapt to changes in topological environment and network load, an improved low delay Q-routing algorithm based on full echoes is proposed. The additional learning rate factor, that of the original algorithm, is replaced by the hyperbolic secant operator, which has a larger adjustment range, better adaptability and more robust performance. The learning rate in the improved algorithm could be adjusted adaptively according to different network conditions, which provides reasonable routing decision. The simulation results show that the algorithm can be suitable for static and dynamic topological environment. Compared with the original algorithm, the improved algorithm can effectively reduce the average delivery time of data under high and low loads, cut down on the oscillations between routes, improve the delivery rate of data packets, and reflect better robustness in the medium load.

Key words: self-organizing network, Q-routing, full echo, self-adaption

CLC Number:

TN911

Qingdong HUANG, Runzhi YUAN, Minpeng GUO, Binyu SHI, Yiyuan CAO. Advanced low delay Q-routing algorithm based on full echoes[J]. Systems Engineering and Electronics, 2020, 42(4): 940-947.

Figures/Tables 13

Fig.4

Fig.1

Fig.2

Fig.3

Table 1

Fig.5

Fig.6

Fig.7

Table 2

Fig.8

Fig.9

Table 3

Table 4

References 17

1	KUSHWAH R , TAPASWI S , KUMAR A . A detailed study on internet connectivity schemes for mobile Ad hoc network[J]. Wireless Personal Communications, 2019, 104 (4): 1433- 1471. doi: 10.1007/s11277-018-6093-7
2	SRA P , CHAND S . QoS in mobile ad hoc networks[J]. Wireless Personal Communications, 2019, 105 (4): 1599- 1616. doi: 10.1007/s11277-019-06162-y
3	SU Y , FAN R , FU X , et al. DQELR: an adaptive deep Q-network-based energy-and latency-aware routing protocol design for underwater acoustic sensor networks[J]. IEEE Access, 2019, 7, 9091- 9104. doi: 10.1109/ACCESS.2019.2891590
4	MAMMERI Z . Reinforcement learning based routing in networks: review and classification of approaches[J]. IEEE Access, 2019, 7, 55916- 55950. doi: 10.1109/ACCESS.2019.2913776
5	SHILOVA Y, KAVALEROV M, BEZUKLADNIKOV I I. Full echo Q-routing with adaptive learning rates: a reinforcement learning approach to network routing[C]//Proc.of the IEEE NW Russia Young Researchers in Electrical and Electronic Engineering Conference, 2016: 341-344.
6	KAVALEROV M, SHILOVA Y, LIKHACHEVA Y. Adaptive Q-routing with random echo and route memory[C]//Proc.of the Conference of Open Innovations Association, 2017: 138-145.
7	KAVALEROV M V, SHILOVA Y A, BEZUKLADNIKOV I I. Preventing instability in full echo Q-routing with adaptive learning rates[C]//Proc.of the IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering, 2017: 155-159.
8	KAVALEROV M, LIKHACHEVA Y, SHILOVA Y. A reinforcement learning approach to network routing based on adaptive learning rates and route memory[C]//Proc.of the SoutheastCon, 2017: 1-6.
9	SHILOVA Y A, BEZUKLADNIKOV I I. Influence of the battery life parameter on the Q-routing algorithm results[C]//Proc.of the IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering, 2017: 213-217.
10	LIU Y , LU D , ZHANG G , et al. Q-learning based content placement method for dynamic cloud content delivery networks[J]. IEEE Access, 2019, 7, 66384- 66394. doi: 10.1109/ACCESS.2019.2917564
11	WANG F , FENG R J , CHEN H Y . Dynamic routing algorithm with Q-learning for internet of things with delayed estimator[J]. IOP Conference Series: Earth and Environmental Science, 2019, 234 (1): 012048.
12	HE X , JIANG H , SONG Y , et al. Routing selection with reinforcement learning for energy harvesting multi-hop CRN[J]. IEEE Access, 2019, 7, 54435- 54448. doi: 10.1109/ACCESS.2019.2912996
13	MAYADUNNA H, SILVA S L D, WEDAGE I, et al. Improving trusted routing by identifying malicious nodes in a MANET using reinforcement learning[C]//Proc.of the 11th International Conference on Advances in ICT for Emerging Regions, 2017: 1-8.
14	SARKAR S , DATTA R . Mobility-aware route selection technique for mobile ad hoc networks[J]. IET Wireless Sensor Systems, 2017, 7 (3): 55- 64. doi: 10.1049/iet-wss.2016.0058
15	YANG X , TAN J . Design and implementation of networked control system based on ARM and Zigbee[J]. IOP Conference Series: Earth and Environmental Science, 2019, 252 (5): 052139- 052144.
16	胡君, 叶俊杰. 基于链路时间预测的移动Ad Hoc网路由[J]. 传感技术学报, 2019, 32 (6): 945- 949. doi: 10.3969/j.issn.1004-1699.2019.06.021
	HU J , YE J J . Routing of mobile ad hoc networks based on link time prediction[J]. Chinese Journal of Sensors and Actuators, 2019, 32 (6): 945- 949. doi: 10.3969/j.issn.1004-1699.2019.06.021
17	胡博, 陈侃松, 顾豪爽, 等. 基于网络拓扑变化的改进型按需路由协议[J]. 小型微型计算机系统, 2019, 40 (3): 589- 593. doi: 10.3969/j.issn.1000-1220.2019.03.023
	HU B , CHEN K S , GU H S , et al. Improved on-demand routing protocol based on network topology change[J]. Journal of Chinese Computer Systems, 2019, 40 (3): 589- 593. doi: 10.3969/j.issn.1000-1220.2019.03.023

仿真参数	值
仿真模型	不规则固定拓扑
仿真时间	10 000单位时间
网络负载	服从指数为λ的泊松分布
学习率η	0.9
包的记忆长度L	1
节点队列最大缓存包数	50
k	0.6
k₂	0.3

仿真参数	值
仿真模型	RWM移动模型
仿真区域面积	1 500 m×1 500 m
仿真时间	10 000单位时间
移动模型节点数量	100
节点移动最大速度	10 m/单位时间
传输范围/m	250
k	0.6
k₂	3

算法	各节点对投递率/%					总投递率/%
算法	0~17	1~23	2~29	7~10	8~4	总投递率/%
AQFE-M算法	84.70	84.67	85.31	85.88	87.90	85.33
改进算法	85.52	85.76	87.24	86.75	89.08	86.77

算法	各节点对投递率/%					总投递率/%
算法	0~17	1~23	2~29	7~10	8~4	总投递率/%
AQFE-M算法	61.50	60.47	65.20	60.88	62.04	61.98
改进算法	64.87	60.89	67.67	62.38	63.12	63.81

Advanced low delay Q-routing algorithm based on full echoes

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 17

Related Articles 1

Recommended Articles

Metrics

Comments