跳到主要导航 跳到搜索 跳到主要内容

DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices

  • Guanyu Xu
  • , Zhiwei Hao
  • , Yong Luo
  • , Han Hu*
  • , Jianping An
  • , Shiwen Mao
  • *此作品的通讯作者
  • Beijing Inst Technol
  • Auburn Univ

科研成果: 期刊稿件 › 论文 › 同行评审

60 引用 (Scopus)

摘要

Recent years have witnessed the great success of vision transformer (ViT), which has achieved state-of-the-art performance on multiple computer vision benchmarks. However, ViT models suffer from vast amounts of parameters and high computation cost, leading to difficult deployment on resource-constrained edge devices. Existing solutions mostly compress ViT models to a compact model but still cannot achieve real-time inference. To tackle this issue, we propose to explore the divisibility of transformer structure, and decompose the large ViT into multiple small models for collaborative inference at edge devices. Our objective is to achieve fast and energy-efficient collaborative inference while maintaining comparable accuracy compared with large ViTs. To this end, we first propose a collaborative inference framework termed DeViT to facilitate edge deployment by decomposing large ViTs. Subsequently, we design a decomposition-and-ensemble algorithm based on knowledge distillation, termed DEKD, to fuse multiple small decomposed models while dramatically reducing communication overheads, and handle heterogeneous models by developing a feature matching module to promote the imitations of decomposed models from the large ViT. Extensive experiments for three representative ViT backbones on four widely-used datasets demonstrate our method achieves efficient collaborative inference for ViTs and outperforms existing lightweight ViTs, striking a good trade-off between efficiency and accuracy. For example, our DeViTs improves end-to-end latency by 2.89x with only 1.65% accuracy sacrifice using CIFAR-100 compared to the large ViT, ViT-L/16, on the GPU server. DeDeiTs surpasses the recent efficient ViT, MobileViT-S, by 3.54% in accuracy on ImageNet-1 K, while running 1.72x faster and requiring 55.28% lower energy consumption on the edge device.
源语言英语
页(从-至)5917-5932
页数16
期刊IEEE Transactions on Mobile Computing
卷23
期5
早期在线日期1月 2023
DOI
出版状态已出版 - 5月 2024

基金资助

No Statement Available

联合国可持续发展目标

此成果有助于实现下列可持续发展目标:

  1. 可持续发展目标 7 - 经济适用的清洁能源
    可持续发展目标 7 经济适用的清洁能源

关键词

  • Transformers
  • Computational modeling
  • Collaboration
  • Servers
  • Costs
  • Graphics processing units
  • Image edge detection
  • Collaborative inference
  • edge computing
  • model decomposition
  • vision transformer

收录

  • EI
  • Scopus
  • SCIE

学术指纹

探究 'DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices' 的科研主题。它们共同构成独一无二的学术指纹。

引用此