基于MADDPG的分布式测控网络群切换算法

doi:10.12305/j.issn.1001-506X.2025.06.31

摘要/Abstract

摘要：

受到战争等特殊环境下部分节点导航拒止、节点移动性与环境干扰所带来的影响, 快速进行测控网络拓扑重构是保证连续测控关键。为了解决上述问题, 针对多体制无人集群测控网络的场景, 提出一种基于多智能体深度确定性策略梯度(multi-agent deep deterministic policy gradient, MADDPG)的分布式多智能体测控网络群切换算法。该算法运用局部可观测马尔可夫决策模型, 并考虑最小连通度、能耗与测控精度设计奖励函数, 构建可靠的测控定位系统。仿真结果表明, 该算法在不同的干扰环境下能有效抵抗外界干扰, 保证测控定位的正常运行, 与传统切换算法相比切换成功率提升12%以上。

关键词: 拓扑重构, 群切换, 测控定位, 局部可观测马尔可夫决策, 最小连通度

Abstract:

Affected by the navigation denial in special environments such as war, node mobility and environmental interference, rapid topology reconfiguration of the measurement and control network is the key to ensure continuous positioning. In order to solve the above problems, a distributed multi-agent measurement and control network group switching algorithm based on multi-agent deep deterministic policy gradient (MADDPG) for the scenario of multi system unmanned cluster measurement and control network is proposed. This algorithm employs a partially observable Markov decision process model and considers minimum connectivity, energy consumption and measurement and control precision to design the reward function, and constructs a reliable measurement and control localization system. Simulation results show that the algorithm is feasible and effective in different interference environments to effectively resist external interference, ensure the normal operation of measurement, control and localization, and improve the switching success rate to more than 12% compared with traditional switching algorithms.

Key words: topology reconfiguration, group switching, measurement and control positioning, partially observable Markov decision, minimum connectivity

中图分类号:

TN393

王慧琳, 刘胜利, 谢岸宏, 朱立东. 基于MADDPG的分布式测控网络群切换算法[J]. 系统工程与电子技术, 2025, 47(6): 2036-2046.

Huilin WANG, Shengli LIU, Anhong XIE, Lidong ZHU. Group switching algorithm for distributed measurement and control network based on MADDPG[J]. Systems Engineering and Electronics, 2025, 47(6): 2036-2046.

图/表 15

图1

图2

图3

图4

图5

图6

图7

图8

表1

表2

图9

图10

图11

图12

图13

参考文献 32

1	BISHNU A , GHOSH A , KOALY S , et al. Small vertex cover helps in fixed-parameter tractability of graph deletion problems over data streams[J]. Theory of Computing Systems, 2023, 67 (6): 1241- 1267. doi: 10.1007/s00224-023-10136-w
2	ANDRADE E , DAHL G . Combinatorial Fiedler theory and graph partition[J]. Linear Algebra and its Applications, 2024, 687, 229- 251. doi: 10.1016/j.laa.2024.02.005
3	SIMCJPWOTZ M, FOSTER D. Naive exploration is optimal for online LQR[C]//Proc. of the International Conference on Machine Learning, 2020: 8937-8948.
4	ROOHI M , ZHANG C , TAHERI M , et al. Synchronization of fractional-order delayed neural networks using dynamic-free adaptive sliding mode control[J]. Fractal and Fractional, 2023, 7 (9): 682. doi: 10.3390/fractalfract7090682
5	DE-CARLI N, SALARIS P, GIORDANO P R. Distributed control barrier functions for global connectivity maintenance[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2024.
6	AKRAM V K , DAGDEVIREN O , TAVLI B . A coverage-aware distributed k-connectivity maintenance algorithm for arbitrarily large k in mobile sensor networks[J]. IEEE/ACM Trans. on Networking, 2021, 30 (1): 62- 75.
7	ZHAO J D, SUN S W, SHEN H, et al. An effective network repair strategy against both random and malicious edge attacks[C]// Proc. of the 40th Chinese Control Conference, 2021: 8628-8633.
8	XIAO X B, WANG Y, HE X M, et al. Restoration optimization method of fault repair path in distribution network under extreme events[C]//Proc. of the IEEE Industrial and Commercial Power System, 2022: 1731-1735.
9	CHEN G Y , SUN P , ZHANG J Y . Repair strategy of military communication network based on discrete artificial bee colony algorithm[J]. IEEE Access, 2020, 8, 73051- 73060. doi: 10.1109/ACCESS.2020.2987860
10	ZHAO B M, XU P, SHI Y X, et al. Preference-aware task assignment in on-demand taxi dispatching: an online stable matching approach[C]//Proc. of the Advancement of Artificial Intelligence Conference on Artificial Intelligence, 2019.
11	WEI X L , YANG L F , CAO G , et al. Recurrent MADDPG for object detection and assignment in combat tasks[J]. IEEE Access, 2020, 8, 163334- 163343. doi: 10.1109/ACCESS.2020.3022638
12	XU D , CHEN G . Autonomous and cooperative control of UAV cluster with multi-agent reinforcement learning[J]. The Aeronautical Journal, 2022, 126 (1300): 932- 951. doi: 10.1017/aer.2021.112
13	ZHANG S Y , XU R H , XU Z B , et al. Dynamic routing of multiple QoS-required flows in cloud-edge autonomous multi-domain data center networks[J]. Computers, Materials & Continua, 2024, 78 (2): 2287- 2308.
14	周子龙, 周杰, 罗宏, 等. 基于局部自适应特征加权算法的遥感图像目标检测[J]. 遥感技术与应用, 2024, 39 (3): 633- 641.
	ZHOU Z L , ZHOU J , LUO H , et al. Remote sensing image target detection based on local adaptive feature weighting algorithm[J]. Remote Sensing Technology and Application, 2024, 39 (3): 633- 641.
15	RAMI K P , SREEDEVI P , POOMIMA E , et al. FTOR-Mod PSO: a fault tolerance and an optimal relay node selection algorithm for wireless sensor networks using modified PSO[J]. Knowledge-Based Systems, 2023, 272, 110583. doi: 10.1016/j.knosys.2023.110583
16	LAZO J , WATTS D . Stochastic model for active distribution networks planning: an analysis of the combination of active network management schemes[J]. Renewable and Sustainable Energy Reviews, 2024, 191, 114156. doi: 10.1016/j.rser.2023.114156
17	MORAGREGA A , FERNMANDEZ-PRADES C . A data-driven factor graph model for anchor-based positioning[J]. Sensors, 2023, 23 (12): 5660. doi: 10.3390/s23125660
18	ALHAFID K , YOUNIS S , MOHAMMED ALI Y E . Efficient near-field localization aided with reconfigurable intelligent surface using geometric dilution of precision[J]. Journal of Information and Telecommunication, 2024, 8 (2): 208- 230. doi: 10.1080/24751839.2023.2272488
19	LIANG W , ZHANG Z Q , ZHANG Y , et al. Improved social spider algorithm for partial disassembly line balancing problem considering the energy consumption involved in tool switching[J]. International Journal of Production Research, 2023, 61 (7): 2250- 2266. doi: 10.1080/00207543.2022.2069059
20	高思华, 刘宝煜, 惠康华, 等. 信息年龄约束下的无人机数据采集能耗优化路径规划算法[J]. 电子与信息学报, 2024, 46 (10): 4024- 4034. doi: 10.11999/JEIT240075
	GAO S H , LIU B Y , HUI K H , et al. Optimized path planning algorithm for energy consumption optimization of UAV data acquisition under information age constraints[J]. Journal of Electronics & Information Technology, 2024, 46 (10): 4024- 4034. doi: 10.11999/JEIT240075
21	RILEY D , EINAY I , GUILLARD F . A constitutive model for porous media with recurring stress drops: from snow to foams and cereals[J]. International Journal of Solids and Structures, 2023, 262, 112044.
22	ZHANG K , YANG Z L , LIANG Q M , et al. China's carbon emissions and energy demand under different methods of global mitigation cooperation: application of an extended RICE model with energy details[J]. Energy, 2023, 285, 129290. doi: 10.1016/j.energy.2023.129290
23	NURWARSITO H, AZIZ A R. Implementation of the Friis free space propagation model in the dynamic source routing (DSR) routing protocol in the vehicular Ad-hoc network (VANET) with variations of road models[C]//Proc. of the Journal of Physics: Conference Series, 2021, 1962(1): 012063.
24	LI B , WANG J M , SONG C , et al. Multi-UAV roundup strategy method based on deep reinforcement learning CEL-MADDPG algorithm[J]. Expert Systems with Applications, 2024, 245, 123018. doi: 10.1016/j.eswa.2023.123018
25	LIU Q , SZEPESVARI C , JIN C . Sample-efficient reinforcement learning of partially observable Markov games[J]. Advances in Neural Information Processing Systems, 2022, 35, 18296- 18308.
26	LIU X , RONG H , NERI P Z G . Deep deterministic policy gradient with constraints for gait optimisation of biped robots[J]. Integrated Computer-Aided Engineering, 2024, 31 (2): 139- 156. doi: 10.3233/ICA-230724
27	HAMEED M K , IDREES A K . Energy-aware scheduling protocol-based hybrid metaheuristic technique to optimize the lifespan in WSNs[J]. The Journal of Supercomputing, 2024, 80 (9): 12706- 12726. doi: 10.1007/s11227-024-05921-4
28	VISHNEVETSKIV K S . Comparison of purely greedy and orthogonal greedy algorithm[J]. Mathematical Notes, 2024, 115 (1/2): 37- 43.
29	CHEN Y Q , HAN S D , CHEN G H , et al. A deep reinforcement learning-based wireless body area network offloading optimization strategy for healthcare services[J]. Health Information Science and Systems, 2023, 11 (1): 8. doi: 10.1007/s13755-023-00212-3
30	JIN X Y , LI H J , YU D X , et al. Topological optimization of continuous action iterated dilemma based on finite-time strategy using DQN[J]. Pattern Recognition Letters, 2024, 182, 133- 139. doi: 10.1016/j.patrec.2024.04.010
31	杨喆, 邓立宝, 狄原竹, 等. 超密集异构无线网络中基于移动轨迹预测的网络切换算法[J]. 电子与信息学报, 2023, 45 (11): 4281- 4291.
	YANG J , DENG L B , DI Y Z , et al. Network switching algorithm based on mobile trajectory prediction in ultra-dense hete-rogeneous wireless networks[J]. Journal of Electronics & Information Technology, 2023, 45 (11): 4281- 4291.
32	TALEKAR S , BANAIT S , PATIL M . Improved Q-reinforcement learning based optimal channel selection in cognitive radio networks[J]. International Journal of Computer Networks & Communications, 2023, 15 (3): 140- 152.

网络类型	速度/(km/min)	相对高度/km
低轨卫星	-	1 200
高空气球	[20, 30]	[200, 210]
无人机	[15, 30]	[50, 60]
无人飞艇	[5, 8]	[10, 20]
无人车	0	0
无人船	0	0

网络参数	符号	数值
天线发射功率/mW	P_t	60
光速/(m/s)	c	3×10⁸
载波频率/GHz	f	23
传输速率/Mbps	R	11
数据包大小/bit	B	512
天线转向能耗/(J/(°))	γ	0.001
其他固有的能源消耗/J	E_u^c(t)	80

[1]	宋怡成, 齐瑞云. 通信故障下基于一跳无人机编队拓扑重构[J]. 系统工程与电子技术, 2025, 47(5): 1706-1717.
[2]	史朝卫, 孟相如, 康巧燕, 苏玉泽. 基于混合流量预测的虚拟网络拓扑重构方法[J]. 系统工程与电子技术, 2021, 43(5): 1382-1388.
[3]	杨文, 王钺, 石金明, 袁坚. 基于多跳位置估计的无线光自组网拓扑重构[J]. 系统工程与电子技术, 2020, 42(2): 452-457.
[4]	林晋福，柏鹏，林志国，刘铭. 基于链路保持时间的认知移动自组网拓扑算法[J]. 系统工程与电子技术, 2014, 36(4): 746-751.