基于MADDPG算法的星地协同边缘计算任务卸载方法

doi:10.12305/j.issn.1001-506X.2026.01.31

系统工程与电子技术 ›› 2026, Vol. 48 ›› Issue (1): 350-360.doi: 10.12305/j.issn.1001-506X.2026.01.31

基于MADDPG算法的星地协同边缘计算任务卸载方法

宋传龙¹^,²(), 张倩武¹(), 何健²^,*(), 周文骏¹^,²(), 王辉²(), 孔巍巍²(), 田文波²

1. 上海大学通信与信息工程学院，上海 200444
2. 上海航天电子通讯设备研究所上海市天基异构网络协同计算重点实验室，上海 201100

收稿日期:2024-05-29 出版日期:2026-01-25 发布日期:2026-02-11
通讯作者: 何健 E-mail:1285635433@qq.com;zhangqianwu@shu.edu.cn;hejian@sina.com;1547283254@qq.com;wanghuiahu@126.com;kongweiwei@804.sast.casc
作者简介:宋传龙（2000—），男，硕士研究生，主要研究方向为天基边缘计算、任务卸载
张倩武（1984—），男，副教授，博士，主要研究方向为光纤通信、宽带光接入网、物联网技术、光通信测试仪器仪表、复杂网络协议分析
周文骏（1999—），男，硕士研究生，主要研究方向为天基边缘计算、边缘特征融合
王　辉（1992—），男，工程师，博士，主要研究方向为移动边缘计算、深度强化学习
孔巍巍（1980—），女，工程师，硕士，主要研究方向为卫星通信网络、边缘计算
田文波（1979—），男，研究员，硕士，主要研究方向为移动边缘计算、星载计算机系统架构
基金资助:
国家重点研发计划（2023YFE0208100）资助课题

Edge computing task offloading method of satellite-ground collaborative based on MADDPG algorithm

Chuanlong SONG¹^,²(), Qianwu ZHANG¹(), Jian HE²^,*(), Wenjun ZHOU¹^,²(), Hui WANG²(), Weiwei KONG²(), Wenbo TIAN²

1. School of Communication & Information Engineering，Shanghai University，Shanghai 200444，China
2. Shanghai Key Laboratory of Intelligent Computing for Spatial Heterogeneous Network, Shanghai Aerospace Electronic Technology Institute，Shanghai 201100，China

Received:2024-05-29 Online:2026-01-25 Published:2026-02-11
Contact: Jian HE E-mail:1285635433@qq.com;zhangqianwu@shu.edu.cn;hejian@sina.com;1547283254@qq.com;wanghuiahu@126.com;kongweiwei@804.sast.casc

摘要/Abstract

摘要：

低轨卫星网络作为空间通信网络的重要组成部分，能够有效弥补地面通信网络在边缘计算方面存在的地面基站覆盖面较低、易受自然灾害影响等问题。因此，基于地面边缘计算场景，引入低轨卫星网络，构建了星地协同网络边缘计算场景，并对边缘计算中的关键技术任务卸载进行了研究。鉴于星地网络中通信距离较长、星上资源有限以及各节点相对位置变化迅速等因素可能导致的系统时延和能耗增加问题，以最小化系统时延和能耗为目标，将其建模为一个马尔可夫博弈过程。提出了一种针对星地协同网络边缘计算场景的基于多智能体深度确定性策略梯度的强化学习算法，通过集中式训练和分布式执行来降低系统综合开销。最后，将该算法与其他算法进行比较，结果表明，相较于任务本地处理、深度确定性策略梯度算法和双延迟深度确定性策略梯度算法，该算法使系统综合开销平均降低了37.25%、9.420%和6.756%，同时具备更好的稳定性和收敛性。

关键词: 星地协同网络, 边缘计算, 深度强化学习, 任务卸载决策

Abstract:

Low Earth orbit satellite networks serve as crucial components of space communication networks, effectively addressing challenges encountered by terrestrial communication networks in edge computing, such as limited coverage of ground-based stations and vulnerability to natural disasters. Therefore, this study introduces low Earth orbit satellite networks into the context of ground edge computing, forming a satellite-ground cooperative network for edge computing scenarios and investigating key tasks offloading in edge computing. Given that factors like long communication distances, limited satellite resources, and rapid changes in node positions in satellite-ground networks can lead to increased system latency and energy consumption, this paper aims to minimize system latency and energy consumption by modeling them as a Markov game. A reinforcement learning algorithm based on multi-agent deep deterministic policy gradient specifically tailored for satellite-ground cooperative network edge computing scenarios is proposed, aiming to reduce overall system overhead through centralized training and distributed execution. Finally comparative analysis with other algorithms demonstrates that, compared to local task processing, deep deterministic policy gradient algorithm, and double-delay deep deterministic policy gradient algorithm, the proposed algorithm achieves an average reduction in system overhead of 37.25%, 9.420%, and 6.756%, respectively, while exhibiting superior stability and convergence.

Key words: satellite-ground collaborative network, edge computing, deep reinforcement learning, task offloading decision

中图分类号:

TN 927

宋传龙, 张倩武, 何健, 周文骏, 王辉, 孔巍巍, 田文波. 基于MADDPG算法的星地协同边缘计算任务卸载方法[J]. 系统工程与电子技术, 2026, 48(1): 350-360.

Chuanlong SONG, Qianwu ZHANG, Jian HE, Wenjun ZHOU, Hui WANG, Weiwei KONG, Wenbo TIAN. Edge computing task offloading method of satellite-ground collaborative based on MADDPG algorithm[J]. Systems Engineering and Electronics, 2026, 48(1): 350-360.

图/表 13

图1

图2

图3

表1

图4

图5

图6

表2

图7

图8

图9

图10

图11

参考文献 30

1	CASONI M, GRAZIA C A, KLAPEZ M, et al. Integration of satellite and LTE for disaster recovery[J]. IEEE Communications Magazine, 2015, 53 (3): 47- 53. doi: 10.1109/MCOM.2015.7060481
2	FANG J, MA A N. IoT application modules placement and dynamic task processing in edge-cloud computing[J]. IEEE Internet of Things Journal, 2021, 8 (16): 12771- 12781. doi: 10.1109/JIOT.2020.3007751
3	MAO Y Y, YOU C S, ZHANG J , et al. A survey on mobile edge computing: the communication perspective[J]. IEEE Communications Surveys and Tutorials, 2017, 19(4): 2322−2358.
4	CHIANG M, ZHANG T. Fog and IoT: an overview of research opportunities[J]. IEEE Internet of Things Journal, 2016, 3 (6): 854- 864. doi: 10.1109/JIOT.2016.2584538
5	RANAWEERA P, JURCUT A, LIYANAGE M. MEC-enabled 5G use cases: a survey on security vulnerabilities and countermeasures[J]. ACM Computing Surveys, 2022, 54 (9): 1- 37.
6	SPINELLI F, MANCUSO V. Toward enabled industrial verticals in 5G: a survey on MEC-based approaches to provisioning and flexibility[J]. IEEE Communications Surveys and Tutorials, 2021, 23 (1): 596- 630. doi: 10.1109/COMST.2020.3037674
7	谢人超, 廉晓飞, 贾庆民, 等. 移动边缘计算卸载技术综述[J]. 通信学报, 2018, 39 (11): 132- 155.
	XIE R C, LIAN X F, JIA Q M, et al. Survey on computation offloading in mobile edge computing[J]. Journal of Communications, 2018, 39 (11): 132- 155.
8	PFANDZELTER T, BERMBACH D. Celestial: virtual software system testbeds for the LEO edge[EB/OL]. [2024-03-29]. https://arxiv.org/obs/2204.06282.
9	CHIEN W C, LAI C F, HOSSAIN M S, et al. Heterogeneous space and terrestrial integrated networks for IoT: architecture and challenges[J]. IEEE Network, 2019, 33 (1): 15- 21. doi: 10.1109/MNET.2018.1800182
10	MOHAMMED A S, VENKATACHALAM K, HUBáLOVSKý S, et al. Smart edge computing for 5G/6G satellite IOT for reducing inter transmission delay[J]. Mobile Networks and Applications, 2022, 27 (3): 1050- 1059. doi: 10.1007/s11036-021-01860-z
11	苏昭阳, 刘留, 艾渤, 等. 面向低轨卫星的星地信道模型综述[J]. 电子与信息学报, 2024, 46(5): 1684−1702.
	SU Z Y, LIU L, Al B, et al. Survey of satellite-ground channel models for low Earth orbit satellites[J]. Journal of Electronics & Information Technology, 2024, 46(5): 1684−1702.
12	CASSARA P, GOTTA A, MARCHESE M, et al. Orbital edge offloading on mega-LEO satellite constellations for equal access to computing[J]. IEEE Communications Magazine, 2022, 60 (4): 32- 36. doi: 10.1109/MCOM.001.2100818
13	唐琴琴, 谢人超, 刘旭, 等. 融合MEC的星地协同网络: 架构、关键技术与挑战[J]. 通信学报, 2020, 41 (4): 162- 181.
	TANG Q Q, XIE R C, LIU X, et al. MEC enabled satellite-terrestrial network: architecture, key technique and challenge[J]. Journal on Communications, 2020, 41 (4): 162- 181.
14	REN J K, HE Y H, YU G D, et al. Joint communication and computation resource allocation for cloud-edge collaborative system[C]//Proc. of the IEEE Wireless Communications and Networking Conference , 2019.
15	WU H Y, DENG S G, LI W, et al. Request dispatching for minimizing service response time in edge cloud systems[C]//Proc. of the 27th International Conference on Computer Communication and Networks , 2018.
16	XIE Y, ZHU Y W, WANG Y, et al. A novel directional and non-local-convergent particle swarm optimization based workflow scheduling in cloud-edge environment[J]. Future Generation Computer Systems, 2019, 97, 361- 378. doi: 10.1016/j.future.2019.03.005
17	TANG Q Q, FEI Z S, LI B, et al. Computation offloading in LEO satellite networks with hybrid cloud and edge computing[J]. IEEE Internet of Things Journal, 2021, 8 (11): 9164- 9176. doi: 10.1109/JIOT.2021.3056569
18	ZHANG Z J, ZHANG W Y, TSENG F H. Satellite mobile edge computing: improving QoS of high-speed satellite-terrestrial networks using edge computing techniques[J]. IEEE Network, 2019, 33 (1): 70- 76. doi: 10.1109/MNET.2018.1800172
19	ZHU X M, JIANG C X. Delay optimization for cooperative multi-tier computing in integrated satellite-terrestrial networks[J]. IEEE Journal on Selected Areas in Communications, 2023, 41 (2): 366- 380. doi: 10.1109/JSAC.2022.3227083
20	DONG F H, HUANG T, ZHANG Y S, et al. A computation offloading strategy in LEO constellation edge cloud network[J]. Electronics （Basel）, 2022, 11 (13): 2024.
21	CAO S H, CHEN S, CHEN H, et al. HCOME: research on hybrid computation offloading strategy for MEC based on DDPG[J]. Electronics （Basel）, 2023, 12 (3): 562.
22	HOU Y N, LI L F, WEI Q, et al. A novel DDPG method with prioritized experience replay[C]//Proc. of the IEEE International Conference on Systems, Man, and Cybernetics , 2017: 316–321.
23	CHEN W, CHEN Y H, WU J X, et al. A multi-user service migration scheme based on deep reinforcement learning and SDN in mobile edge computing[J]. Physical Communication, 2021, 47, 101397. doi: 10.1016/j.phycom.2021.101397
24	SONG Z Y, HAO Y, LIU Y Y, et al. Energy-efficient multiaccess edge computing for terrestrial-satellite Internet of Things[J]. IEEE Internet of Things Journal, 2021, 8 (18): 14202- 14218. doi: 10.1109/JIOT.2021.3068141
25	WANG Y X, YANG J, GUO X Y, et al. A game-theoretic approach to computation offloading in satellite edge computing[J]. Access, 2020, 8, 12510- 12520. doi: 10.1109/ACCESS.2019.2963068
26	LOWE R, WU Y, AVIV T, et al. Multi-agent actor-critic for mixed cooperative-competitive environments[C]//Proc. of the Advances in Neural Information Processing Systems, 2017.
27	CAO X L, YANG B, SHEN Y L, et al. Edge-assisted multi-layer offloading optimization of LEO satellite-terrestrial integrated networks[J]. IEEE Journal on Selected Areas in Communications, 2023, 41 (2): 381- 398. doi: 10.1109/JSAC.2022.3227032
28	LAKEW D S, TRAN A T, DAO N N, et al. Intelligent self-optimization for task offloading in LEO-MEC-assisted energy-harvesting-UAV systems[J]. IEEE Trans. on Network Science and Engineering, 2024, 11 (6): 5135- 5148. doi: 10.1109/TNSE.2023.3349321
29	LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[P]. US: 201615217758, 2020-09-15.
30	CHEN Y L, AI B, NIU Y, et al. Energy-constrained computation offloading in space-air-ground integrated networks using distributionally robust optimization[J]. IEEE Trans. on Vehicular Technology, 2021, 70 (11): 12113- 12125. doi: 10.1109/TVT.2021.3116593

参数	数值
UE数量i	$ i\in \left(\mathrm{2,10}\right) $
用户端计算频率$f_i^u/{\mathrm{GHZ}} $	0.3
BS计算频率$f_i^{{\mathrm{BS}}}/{\mathrm{GHZ}} $	20
LEO计算频率$f_i^{{\mathrm{sat}}}/{\mathrm{GHZ}} $	30
BS任务传输速率$ R\mathrm{_{BS}}/\mathrm{Mbps} $	1.5
LEO任务传输速率$ R\mathrm{_\text{sat}}/\mathrm{Mbps} $	15
BS任务传输功率$ P_{i,\text{trans}}^{\mathrm{BS}}/\mathrm{W} $	40
LEO任务传输功率$P_{i,\text{trans}}^{{\mathrm{sat}}}/{\mathrm{W}} $	630
BS覆盖半径${R}_\text{Bs}/{\mathrm{m}} $	500

参数	符号
$a \in A$表示第a层Actor网络	a
$c \in C$表示第c层Critic网络	c
Actor网络网络层数	$ {L_a} $
Critic网络网络层数	${L_c}$
Actor网络神经元数量	$ {N_a} $
Critic网络神经元数量	$ {N_c} $
一次训练取样数据样本数量	${\mathrm{Batch}} - {\mathrm{size}}$
输出维度	${\mathrm{act}} - \dim $

[1]	薛锦妍, 张雅声, 陶雪峰, 杨茗棋, 赵帅龙. GEO航天器轨道机动控制研究进展[J]. 系统工程与电子技术, 2026, 48(1): 290-300.
[2]	魏潇龙, 吴亚荣, 姚登凯, 赵顾颢. 基于深度强化学习的无人机空战机动分层决策算法[J]. 系统工程与电子技术, 2025, 47(9): 2993-3003.
[3]	朱运豆, 孙海权, 胡笑旋. 基于指针网络架构的多星协同成像任务规划方法[J]. 系统工程与电子技术, 2025, 47(7): 2246-2255.
[4]	孟麟芝, 孙小涓, 胡玉新, 高斌, 孙国庆, 牟文浩. 面向卫星在轨处理的强化学习任务调度算法[J]. 系统工程与电子技术, 2025, 47(6): 1917-1929.
[5]	郑康洁, 张新宇, 王伟菘, 刘震生. DQN与规则结合的智能船舶动态自主避障决策[J]. 系统工程与电子技术, 2025, 47(6): 1994-2001.
[6]	刘书含, 李彤, 李富强, 杨春刚. 意图态势双驱动的数据链抗干扰通信机制[J]. 系统工程与电子技术, 2025, 47(6): 2055-2064.
[7]	熊威, 张栋, 任智, 杨书恒. 面向有人/无人机协同打击的智能决策方法研究[J]. 系统工程与电子技术, 2025, 47(4): 1285-1299.
[8]	马鹏, 蒋睿, 王斌, 徐盟飞, 侯长波. 基于隐式对手建模的策略重构抗智能干扰方法[J]. 系统工程与电子技术, 2025, 47(4): 1355-1363.
[9]	潘沭伽, 陈思光. 面向资源异构的通信高效去中心化联邦学习[J]. 系统工程与电子技术, 2025, 47(4): 1374-1383.
[10]	唐开强, 傅汇乔, 刘佳生, 邓归洲, 陈春林. 基于深度强化学习的带约束车辆路径分层优化研究[J]. 系统工程与电子技术, 2025, 47(3): 827-841.
[11]	陈夏瑢, 李际超, 陈刚, 刘鹏, 姜江. 基于异质网络的装备体系组合发展规划问题[J]. 系统工程与电子技术, 2025, 47(3): 855-861.
[12]	刘洋, 孟凡一, 陈刚. 基于强化学习的变形飞行器抗扰补偿控制方法[J]. 系统工程与电子技术, 2025, 47(12): 4130-4142.
[13]	张耀中, 吴卓然, 张建东, 杨啟明, 史国庆, 徐自祥. 基于ME-DDPG算法的无人机多对一追逃博弈[J]. 系统工程与电子技术, 2025, 47(10): 3288-3299.
[14]	隋东, 蔡向嵘. 智能飞行冲突解脱算法的持续学习机制[J]. 系统工程与电子技术, 2025, 47(10): 3300-3312.
[15]	张庭瑜, 曾颖, 李楠, 黄洪钟. 基于深度强化学习的航天器功率-信号复合网络优化算法[J]. 系统工程与电子技术, 2024, 46(9): 3060-3069.

基于MADDPG算法的星地协同边缘计算任务卸载方法

Edge computing task offloading method of satellite-ground collaborative based on MADDPG algorithm

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 30

相关文章 15

编辑推荐

Metrics

本文评价