面向资源异构的通信高效去中心化联邦学习

doi:10.12305/j.issn.1001-506X.2025.04.34

摘要/Abstract

摘要：

为缓解去中心化联邦学习中不同终端节点数据异构带来的负面影响, 在提升系统异步兼容性的同时降低整体通信开销, 提出一种基于掩码位置图的去中心化联邦学习算法。具体地, 设计一种不对称的掩码更新方案。通过逐渐提升稀疏度, 将掩码范数与训练程度绑定, 同时使用可信任的稀疏联邦聚合, 在有效利用稀疏梯度的同时保障系统的安全性。其次, 设计动态掩码社区分割算法, 将梯度掩码与社区分割结合, 可有效利用全网梯度之间的相似性, 主动选择相似聚合目标, 提升模型性能。进一步, 在网络结构上将模型层与掩码层分离, 可降低算力异构对系统可拓展性的影响。最后, 设计一种单线程、可同时模拟数据异构、算力异构与终端节点异步的实验方案。实验结果表明, 与现有相关方法相比, 所提算法在两种数据集与严苛异步条件设置下均可维持高准确率, 并且将通信开销降低了14%~21%。

关键词: 边缘计算, 联邦学习, 去中心化系统, 稀疏训练

Abstract:

In order to alleviate the negative impact caused by data heterogeneity of different terminal nodes in decentralized federation learning, and to enhance asynchronous compatibility while reducing the overall communication overhead, a decentralized federated learning algorithm based on mask location graph is proposed. Specifically, an asymmetric mask updating scheme is designed, which binds the mask norm to the training degree by gradually increasing the sparsity. It can effectively utilize the sparse gradient and guarantee system security while using trusted sparse federated aggregation. Secondly, a dynamic mask community segmentation algorithm is designed to combine gradient mask with community segmentation, which can effectively utilize the similarity between gradients across the entire network, while actively selecting similar aggregation targets and improving model performance. Furthermore, separating the model layer from the mask layer in the network structure can reduce the impact of arithmetic heterogeneity on the system scalability. Finally, a single-threaded based experimental scheme is developed to simulate data heterogeneity, computility heterogeneity and terminal node asynchrony simultaneously. Experimental results show that compared with existing relevant methods, the proposed algorithm maintains high accuracy in both two datasets and strict asynchronous condition settings, and reduces communication overhead by 14%~21%.

Key words: edge computing, federated learning, decentralized system, sparse training

中图分类号:

TP393

潘沭伽, 陈思光. 面向资源异构的通信高效去中心化联邦学习[J]. 系统工程与电子技术, 2025, 47(4): 1374-1383.

Shujia PAN, Siguang CHEN. Communication-efficient decentralized federated learning with resource heterogeneity[J]. Systems Engineering and Electronics, 2025, 47(4): 1374-1383.

图/表 7

图1

图2

图3

表1

表2

图4

图5

参考文献 30

1	MANOGARANG,SHAKEELP,FOUADH,et al.Wearable IoT smart-log patch: an edge computing-based Bayesian deep learning network system for multi access physical monitoring system[J].Sensors,2019,19(13):3030-3048. doi: 10.3390/s19133030
2	MCENROEP,WANGS,LIYANAGEM.A survey on the convergence of edge computing and AI for UAVs: opportunities and challenges[J].IEEE Internet of Things Journal,2022,9(17):15435-15459. doi: 10.1109/JIOT.2022.3176400
3	LIUY,YUH M,XIES L,et al.Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks[J].IEEE Trans.on Vehicular Technology,2019,68(11):11158-11168. doi: 10.1109/TVT.2019.2935450
4	MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]// Proc. of the International Conference on Artificial Intelligence and Statistics, 2017: 1273-1282.
5	SATTLERF,MULLERK R,SAMEKW.Clustered federated learning: model-agnostic distributed multitask optimization under privacy constraints[J].IEEE Trans.on Neural Networks and Learning Systems,2021,32(8):3710-3722. doi: 10.1109/TNNLS.2020.3015958
6	CHOSHA,CHUNGJ,YIND,et al.An efficient framework for clustered federated learning[J].IEEE Trans.on Information Theory,2022,68(12):8076-8091. doi: 10.1109/TIT.2022.3192506
7	朱素霞,顾玢珂,孙广路.基于相似度加速的自适应聚类联邦学习[J].通信学报,2024,45(3):197-207.
	ZHUS X,GUF K,SUNG L.Adaptive clustering federated learning via similarity acceleration[J].Journal on Communications,2024,45(3):197-207.
8	LI X X, JIANG M R, ZHANG X F, et al. FedBN: learning federated learning on non-IID features via local batch normalization[C]//Proc. of the International Conference on Learning Re-presentations, 2021.
9	XIA Y Y, CHEN S G. Dual aggregated federated learning with depthwise separable convolution for smart healthcare[C]//Proc. of the IEEE International Conference on Communication, 2023: 154-159.
10	ZHUANG W M, WEN Y G, ZANG S. Divergence-aware fede-rated self-supervised learning[C]//Proc. of the International Conference on Learning Representations, 2022.
11	XIE C, KOYEJO O, GUPTA I. Asynchronous Federated Optimization[C]//Proc. of the Neural Information Processing Systems, 2020.
12	赵家雪,苏铓,侯金鹏,等.一种基于同态签名的可验证联邦学习方案[J].密码学报,2023,10(5):1019-1034.
	ZHAOJ X,SUM,HOUJ P,et al.A verifiable federated learning scheme based on homomorphic signatures[J].Journal of Cryptologic Research,2023,10(5):1019-1034.
13	LIY Z,CHENC,LIUN,et al.A blockchain-based decentralized federated learning framework with committee consensus[J].IEEE Network,2021,35(1):234-241. doi: 10.1109/MNET.011.2000263
14	CHEC J,LIX L,CHENC,et al.A decentralized federated learning framework via committee mechanism with convergence guarantee[J].IEEE Trans.on Parallel and Distributed Systems,2022,33(12):4783-4800. doi: 10.1109/TPDS.2022.3202887
15	WANGP,SUNW,ZHANGH B,et al.Distributed and secure federated learning for wireless computing power networks[J].IEEE Trans.on Vehicular Technology,2023,72(7):9381-9393. doi: 10.1109/TVT.2023.3247859
16	FENGL,ZHAOY O,GUOS Y,et al.BAFL: a blockchain-based asynchronous federated learning framework[J].IEEE Trans.on Computers,2022,71(5):1092-1103. doi: 10.1109/TC.2021.3072033
17	KIM Y, HONG C. Blockchain-based node-aware dynamic weighting methods for improving federated learning perfor-mance[C]//Proc. of the Asia-Pacific Network Operations and Management Symposium, 2019.
18	MAJEED U, HONG C. FLchain: federated learning via MEC-enabled blockchain network[C]//Proc. of the Asia-Pacific Network Operations and Management Symposium, 2019.
19	BAO X L, SU C, XIONG Y, et al. Flchain: a blockchain for auditable federated learning with trust and incentive[C]//Proc. of the Big Data Computing and Communications, 2019: 151-159.
20	HONG J Y, WANG H T, WANG Z Y, et al. Efficient split-mix federated learning for on-demand and in-situ customization[C]// Proc. of the International Conference on Learning Representations, 2022.
21	HORVATH S, LASKARIDIS S, ALMEIDE M, et al. FjORD: fair and accurate federated learning under heterogeneous targets with ordered dropout[C]//Proc. of the Neural Information Processing Systems, 2021: 12876-12889.
22	LI A, SUN J W, LI P C, et al. Hermes: an efficient federated learning framework for heterogeneous mobile clients[C]// Proc. of the Mobile Computing and Networking, 2021: 420-437.
23	WANGD,ZHANGN F,TAOM X,et al.Knowledge selection and local updating optimization for federated knowledge distillation with heterogeneous models[J].IEEE Journal of Selected Topics in Signal Processing,2023,17(1):82-97. doi: 10.1109/JSTSP.2022.3223526
24	LIU R X, WU F Z, WU C H, et al. No one left behind: inclusive federated learning over heterogeneous devices[C]//Proc. of the Knowledge Discovery and Data Mining, 2022: 3398-3406.
25	BEILHARZ J, PFITZNER B, SCHMID R, et al. Implicit model specialization through dag-based decentralized federated learning[C]//Proc. of the Middleware Conference, 2021: 310-322.
26	SHI Y F, SHEN L, WEI K, et al. Improving the model consistency of decentralized federated learning[C]//Proc. of the International Conference on Machine Learning, 2023: 31269-31291.
27	KOH,LEEJ,SEOS,et al.Joint client selection and bandwidth allocation algorithm for federated learning[J].IEEE Trans.on Mobile Computing,2023,22(6):3380-3390. doi: 10.1109/TMC.2021.3136611
28	DAI R, SHEN L, HE F X. DisPFL: towards communication-efficient personalized federated learning via decentralized sparse training[C]//Proc. of the International Conference on Machine Learning, 2022.
29	BLONDELD V,GUILLAUMEL J,LAMBIOTTER,et al.Fast unfolding of communities in large networks[J].Journal of Statistical Mechanics: Theory and Experiment,2008,2008,10008. doi: 10.1088/1742-5468/2008/10/P10008
30	HSU T M, QI H, BROWN M. Measuring the effects of non-identical data distribution for federated visual classification[EB/OL]. [2023-10-03]. https://arxiv.org/abs/1909.06335.

数据集	方法	狄利克雷分布	病态数据分布
CIFAR100	Local	33.2	56.4
	FedAvg	49.0	40.3
	DisPFL	54.4	70.6
	FedAsync	47.9	64.5
	本文算法	54.2	69.8
CIFAR10	Local	61.2	85.4
	FedAvg	78.5	56.5
	DisPFL	83.4	89.3
	FedAsync	78.9	87.0
	本文算法	81.0	89.8

方法	最大通信量/MB	平均通信量/MB
Local	-	-
FedAvg	8 534	8 534
DisPFL	8 714	8 534
FedAsync	7 929	7 800
本文算法	13 486	6 746

[1]	刘鹏涛, 雷菁, 刘伟. 无人机边缘计算: 架构、多址接入与计算卸载[J]. 系统工程与电子技术, 2024, 46(9): 3198-3210.
[2]	余雪勇, 傅新程, 朱洪波. 基于混合协作NOMA的安全MEC能耗优化[J]. 系统工程与电子技术, 2024, 46(3): 1116-1124.
[3]	刘世豪, 黄仰超, 胡航, 司江勃, 韩蕙竹, 安琪. 无人机群辅助边缘计算系统的任务卸载和资源分配联合优化[J]. 系统工程与电子技术, 2024, 46(2): 751-760.
[4]	张冰雪, 李希胜, 尤佳, 宋委任. 基于节点复合特性的端-边协同网络生成算法[J]. 系统工程与电子技术, 2023, 45(8): 2588-2596.
[5]	陈韩, 张晶, 董俊, 董洁. 子任务调度和时延联合优化的MEC卸载方案[J]. 系统工程与电子技术, 2023, 45(2): 572-579.
[6]	邹虹, 白陈阳, 何鹏, 崔亚平, 王汝言, 吴大鹏. 基于分布式深度学习的边缘服务放置策略[J]. 系统工程与电子技术, 2022, 44(5): 1728-1737.
[7]	余雪勇, 朱烨, 邱礼翔, 朱洪波. 基于无人机辅助边缘计算系统的节能卸载策略[J]. 系统工程与电子技术, 2022, 44(3): 1022-1029.
[8]	金志刚, 段晨旭, 羊秋玲, 苏毅珊. 基于水下云边协同架构的珊瑚礁监测新机制[J]. 系统工程与电子技术, 2022, 44(12): 3829-3836.
[9]	黄晓舸, 崔艺凡, 张东宇, 陈前斌. 基于MEC的任务卸载和资源分配联合优化方案[J]. 系统工程与电子技术, 2020, 42(6): 1386-1394.