Skip to main navigation Skip to search Skip to main content

TinyFEL: Communication, Computation, and Memory Efficient Tiny Federated Edge Learning via Model Sparse Update

  • Qimei Chen*
  • , Han Cheng
  • , Yipeng Liang
  • , Guangxu Zhu
  • , Ming Li
  • , Hao Jiang
  • *Corresponding author for this work
  • Shenzhen Res Inst Big Data

Research output: Contribution to journal › Article › peer-review

10 Scopus citations

Abstract

Federated edge learning (FEL) is regarded as a promising distributed machine learning paradigm to reduce transmission latency and resources as well as preserve raw data privacy by collaboratively training local deep learning models across multiple edge devices. However, with the development of artificial intelligence (AI) technologies, the size of neural network models grows exponentially with their parameters to meet variable application requirements, which poses significant challenges to the computation, communication, and memory abilities of edge devices. Existing designs typically focus on either communication or computation efficiency without caring each device's memory ability. To deal with the above issues, we first introduce a novel model sparse update enabled tiny FEL (TinyFEL) architecture, which terminates the backpropagation early in local model training processes. Therefore, the proposed TinyFEL can reduce local memory occupation and lessen the communication-and-computation burden. Furthermore, we propose a parameter splitting mechanism instead of transmitting the full model, only a part of updated layers of parameters is transmitted for aggregation, which significantly reduced the communication overheads. Thereafter, we develop a communication and computation latency minimization problem to accelerate the training of TinyFEL. To this end, we theoretically analyze the convergence performance of TinyFEL, which unveils the mathematical relationship among sparse update ratio assignment, device selection, and learning performance. Then, a joint sparse update ratio assignment, device selection, and resource allocation strategy is introduced based on the alternating direction method of multipliers (ADMMs) and block coordinate descent (BCD) algorithms. Numerical results indicate that our proposed TinyFEL can reduce training memory occupation by over 40% than the traditional FEL at the cost of negligible accuracy loss.
Original languageEnglish
Pages (from-to)8247-8260
Number of pages14
JournalIEEE INTERNET OF THINGS JOURNAL
Volume12
Issue number7
Early online dateJan 2024
DOIs
StatePublished - 1 Apr 2025

Funding

This work was supported in part by the Key Research and Development Plan of Hubei Province under Grant 2023BCB041; in part by the Wuhan Science and Technology Achievement Transformation Project under Grant 2024030803010178; in part by the Digital Information Infrastructure Operation and Maintenance Innovation Base Co-raised by Wuhan University, Fiberhome Telecommunication Technologies Company, Ltd, and Oman Telecommunications Company; in part by the National Natural Science Foundation of China under Grant 62371313; in part by the Guangdong Major Project of Basic and Applied Basic Research under Grant 2023B0303000001; in part by the Guangdong Basic and Applied Basic Research Foundation under Grant 2022A1515010109; in part by the Shenzhen-Hong Kong-Macau Technology Research Programme (Type C) under Grant SGDX20230821091559018; and in part by the Longgang District Special Funds for Science and Technology Innovation under Grant LGKCSDPT2023002.

Keywords

  • Computational modeling
  • Training
  • Performance evaluation
  • Memory management
  • Servers
  • Data models
  • Optimization
  • Numerical models
  • Resource management
  • Mathematical models
  • Communication resources
  • computation resources
  • federated edge learning (FEL)
  • model sparse update
  • parameter splitting

Indexed by

  • Scopus
  • EI
  • SCIE

Fingerprint

Dive into the research topics of 'TinyFEL: Communication, Computation, and Memory Efficient Tiny Federated Edge Learning via Model Sparse Update'. Together they form a unique fingerprint.

Cite this