面向资源异构的通信高效去中心化联邦学习

doi:10.12305/j.issn.1001-506X.2025.04.34

Abstract

Abstract:

In order to alleviate the negative impact caused by data heterogeneity of different terminal nodes in decentralized federation learning, and to enhance asynchronous compatibility while reducing the overall communication overhead, a decentralized federated learning algorithm based on mask location graph is proposed. Specifically, an asymmetric mask updating scheme is designed, which binds the mask norm to the training degree by gradually increasing the sparsity. It can effectively utilize the sparse gradient and guarantee system security while using trusted sparse federated aggregation. Secondly, a dynamic mask community segmentation algorithm is designed to combine gradient mask with community segmentation, which can effectively utilize the similarity between gradients across the entire network, while actively selecting similar aggregation targets and improving model performance. Furthermore, separating the model layer from the mask layer in the network structure can reduce the impact of arithmetic heterogeneity on the system scalability. Finally, a single-threaded based experimental scheme is developed to simulate data heterogeneity, computility heterogeneity and terminal node asynchrony simultaneously. Experimental results show that compared with existing relevant methods, the proposed algorithm maintains high accuracy in both two datasets and strict asynchronous condition settings, and reduces communication overhead by 14%~21%.

Key words: edge computing, federated learning, decentralized system, sparse training

CLC Number:

TP393

Shujia PAN, Siguang CHEN. Communication-efficient decentralized federated learning with resource heterogeneity[J]. Systems Engineering and Electronics, 2025, 47(4): 1374-1383.

Figures/Tables 7

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Fig.4

Fig.5

References 30

1	MANOGARANG,SHAKEELP,FOUADH,et al.Wearable IoT smart-log patch: an edge computing-based Bayesian deep learning network system for multi access physical monitoring system[J].Sensors,2019,19(13):3030-3048. doi: 10.3390/s19133030
2	MCENROEP,WANGS,LIYANAGEM.A survey on the convergence of edge computing and AI for UAVs: opportunities and challenges[J].IEEE Internet of Things Journal,2022,9(17):15435-15459. doi: 10.1109/JIOT.2022.3176400
3	LIUY,YUH M,XIES L,et al.Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks[J].IEEE Trans.on Vehicular Technology,2019,68(11):11158-11168. doi: 10.1109/TVT.2019.2935450
4	MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]// Proc. of the International Conference on Artificial Intelligence and Statistics, 2017: 1273-1282.
5	SATTLERF,MULLERK R,SAMEKW.Clustered federated learning: model-agnostic distributed multitask optimization under privacy constraints[J].IEEE Trans.on Neural Networks and Learning Systems,2021,32(8):3710-3722. doi: 10.1109/TNNLS.2020.3015958
6	CHOSHA,CHUNGJ,YIND,et al.An efficient framework for clustered federated learning[J].IEEE Trans.on Information Theory,2022,68(12):8076-8091. doi: 10.1109/TIT.2022.3192506
7	朱素霞,顾玢珂,孙广路.基于相似度加速的自适应聚类联邦学习[J].通信学报,2024,45(3):197-207.
	ZHUS X,GUF K,SUNG L.Adaptive clustering federated learning via similarity acceleration[J].Journal on Communications,2024,45(3):197-207.
8	LI X X, JIANG M R, ZHANG X F, et al. FedBN: learning federated learning on non-IID features via local batch normalization[C]//Proc. of the International Conference on Learning Re-presentations, 2021.
9	XIA Y Y, CHEN S G. Dual aggregated federated learning with depthwise separable convolution for smart healthcare[C]//Proc. of the IEEE International Conference on Communication, 2023: 154-159.
10	ZHUANG W M, WEN Y G, ZANG S. Divergence-aware fede-rated self-supervised learning[C]//Proc. of the International Conference on Learning Representations, 2022.
11	XIE C, KOYEJO O, GUPTA I. Asynchronous Federated Optimization[C]//Proc. of the Neural Information Processing Systems, 2020.
12	赵家雪,苏铓,侯金鹏,等.一种基于同态签名的可验证联邦学习方案[J].密码学报,2023,10(5):1019-1034.
	ZHAOJ X,SUM,HOUJ P,et al.A verifiable federated learning scheme based on homomorphic signatures[J].Journal of Cryptologic Research,2023,10(5):1019-1034.
13	LIY Z,CHENC,LIUN,et al.A blockchain-based decentralized federated learning framework with committee consensus[J].IEEE Network,2021,35(1):234-241. doi: 10.1109/MNET.011.2000263
14	CHEC J,LIX L,CHENC,et al.A decentralized federated learning framework via committee mechanism with convergence guarantee[J].IEEE Trans.on Parallel and Distributed Systems,2022,33(12):4783-4800. doi: 10.1109/TPDS.2022.3202887
15	WANGP,SUNW,ZHANGH B,et al.Distributed and secure federated learning for wireless computing power networks[J].IEEE Trans.on Vehicular Technology,2023,72(7):9381-9393. doi: 10.1109/TVT.2023.3247859
16	FENGL,ZHAOY O,GUOS Y,et al.BAFL: a blockchain-based asynchronous federated learning framework[J].IEEE Trans.on Computers,2022,71(5):1092-1103. doi: 10.1109/TC.2021.3072033
17	KIM Y, HONG C. Blockchain-based node-aware dynamic weighting methods for improving federated learning perfor-mance[C]//Proc. of the Asia-Pacific Network Operations and Management Symposium, 2019.
18	MAJEED U, HONG C. FLchain: federated learning via MEC-enabled blockchain network[C]//Proc. of the Asia-Pacific Network Operations and Management Symposium, 2019.
19	BAO X L, SU C, XIONG Y, et al. Flchain: a blockchain for auditable federated learning with trust and incentive[C]//Proc. of the Big Data Computing and Communications, 2019: 151-159.
20	HONG J Y, WANG H T, WANG Z Y, et al. Efficient split-mix federated learning for on-demand and in-situ customization[C]// Proc. of the International Conference on Learning Representations, 2022.
21	HORVATH S, LASKARIDIS S, ALMEIDE M, et al. FjORD: fair and accurate federated learning under heterogeneous targets with ordered dropout[C]//Proc. of the Neural Information Processing Systems, 2021: 12876-12889.
22	LI A, SUN J W, LI P C, et al. Hermes: an efficient federated learning framework for heterogeneous mobile clients[C]// Proc. of the Mobile Computing and Networking, 2021: 420-437.
23	WANGD,ZHANGN F,TAOM X,et al.Knowledge selection and local updating optimization for federated knowledge distillation with heterogeneous models[J].IEEE Journal of Selected Topics in Signal Processing,2023,17(1):82-97. doi: 10.1109/JSTSP.2022.3223526
24	LIU R X, WU F Z, WU C H, et al. No one left behind: inclusive federated learning over heterogeneous devices[C]//Proc. of the Knowledge Discovery and Data Mining, 2022: 3398-3406.
25	BEILHARZ J, PFITZNER B, SCHMID R, et al. Implicit model specialization through dag-based decentralized federated learning[C]//Proc. of the Middleware Conference, 2021: 310-322.
26	SHI Y F, SHEN L, WEI K, et al. Improving the model consistency of decentralized federated learning[C]//Proc. of the International Conference on Machine Learning, 2023: 31269-31291.
27	KOH,LEEJ,SEOS,et al.Joint client selection and bandwidth allocation algorithm for federated learning[J].IEEE Trans.on Mobile Computing,2023,22(6):3380-3390. doi: 10.1109/TMC.2021.3136611
28	DAI R, SHEN L, HE F X. DisPFL: towards communication-efficient personalized federated learning via decentralized sparse training[C]//Proc. of the International Conference on Machine Learning, 2022.
29	BLONDELD V,GUILLAUMEL J,LAMBIOTTER,et al.Fast unfolding of communities in large networks[J].Journal of Statistical Mechanics: Theory and Experiment,2008,2008,10008. doi: 10.1088/1742-5468/2008/10/P10008
30	HSU T M, QI H, BROWN M. Measuring the effects of non-identical data distribution for federated visual classification[EB/OL]. [2023-10-03]. https://arxiv.org/abs/1909.06335.

数据集	方法	狄利克雷分布	病态数据分布
CIFAR100	Local	33.2	56.4
	FedAvg	49.0	40.3
	DisPFL	54.4	70.6
	FedAsync	47.9	64.5
	本文算法	54.2	69.8
CIFAR10	Local	61.2	85.4
	FedAvg	78.5	56.5
	DisPFL	83.4	89.3
	FedAsync	78.9	87.0
	本文算法	81.0	89.8

方法	最大通信量/MB	平均通信量/MB
Local	-	-
FedAvg	8 534	8 534
DisPFL	8 714	8 534
FedAsync	7 929	7 800
本文算法	13 486	6 746

[1]	Pengtao LIU, Jing LEI, Wei LIU. Unmanned aerial vehicle-enabled edge computing: architecture, multiple access and computation offloading [J]. Systems Engineering and Electronics, 2024, 46(9): 3198-3210.
[2]	Xueyong YU, Xincheng FU, Hongbo ZHU. Optimization of energy consumption with hybrid cooperative NOMA for secure MEC [J]. Systems Engineering and Electronics, 2024, 46(3): 1116-1124.
[3]	Shihao LIU, Yangchao HUANG, Hang HU, Jiangbo SI, Huizhu HAN, Qi AN. Joint optimization of task offloading and resource allocation for UAV swarm-assisted edge computing systems [J]. Systems Engineering and Electronics, 2024, 46(2): 751-760.
[4]	Bingxue ZHANG, Xisheng LI, Jia YOU, Weiren SONG. End-edge collaborative network generation algorithm based on node composite characteristics [J]. Systems Engineering and Electronics, 2023, 45(8): 2588-2596.
[5]	Han CHEN, Jing ZHANG, Jun DONG, Jie DONG. MEC offloading scheme based on joint optimization of subtaskscheduling and delay [J]. Systems Engineering and Electronics, 2023, 45(2): 572-579.
[6]	Hong ZOU, Chenyang BAI, Peng HE, Yaping CUI, Ruyan WANG, Dapeng WU. Edge service placement strategy based on distributed deep learning [J]. Systems Engineering and Electronics, 2022, 44(5): 1728-1737.
[7]	Xueyong YU, Ye ZHU, Lixiang QIU, Hongbo ZHU. Energy efficient offloading strategy for UAV aided edgecomputing systems [J]. Systems Engineering and Electronics, 2022, 44(3): 1022-1029.
[8]	Zhigang JIN, Chenxu DUAN, Qiuling YANG, Yishan SU. A new mechanism for reef coral monitoring based on underwater cloud-edge collaborative architecture [J]. Systems Engineering and Electronics, 2022, 44(12): 3829-3836.
[9]	Xiaoge HUANG, Yifan CUI, Dongyu ZHANG, Qianbin CHEN. Joint optimization scheme of task offloading and resource allocation based on MEC [J]. Systems Engineering and Electronics, 2020, 42(6): 1386-1394.

Communication-efficient decentralized federated learning with resource heterogeneity

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 30

Related Articles 9

Recommended Articles

Metrics

Comments