摘要
Recent years have witnessed the great success of vision transformer (ViT), which has achieved state-of-the-art performance on multiple computer vision benchmarks. However, ViT models suffer from vast amounts of parameters and high computation cost, leading to difficult deployment on resource-constrained edge devices. Existing solutions mostly compress ViT models to a compact model but still cannot achieve real-time inference. To tackle this issue, we propose to explore the divisibility of transformer structure, and decompose the large ViT into multiple small models for collaborative inference at edge devices. Our objective is to achieve fast and energy-efficient collaborative inference while maintaining comparable accuracy compared with large ViTs. To this end, we first propose a collaborative inference framework termed DeViT to facilitate edge deployment by decomposing large ViTs. Subsequently, we design a decomposition-and-ensemble algorithm based on knowledge distillation, termed DEKD, to fuse multiple small decomposed models while dramatically reducing communication overheads, and handle heterogeneous models by developing a feature matching module to promote the imitations of decomposed models from the large ViT. Extensive experiments for three representative ViT backbones on four widely-used datasets demonstrate our method achieves efficient collaborative inference for ViTs and outperforms existing lightweight ViTs, striking a good trade-off between efficiency and accuracy. For example, our DeViTs improves end-to-end latency by 2.89x with only 1.65% accuracy sacrifice using CIFAR-100 compared to the large ViT, ViT-L/16, on the GPU server. DeDeiTs surpasses the recent efficient ViT, MobileViT-S, by 3.54% in accuracy on ImageNet-1 K, while running 1.72x faster and requiring 55.28% lower energy consumption on the edge device.
| 源语言 | 英语 |
|---|---|
| 页(从-至) | 5917-5932 |
| 页数 | 16 |
| 期刊 | IEEE Transactions on Mobile Computing |
| 卷 | 23 |
| 期 | 5 |
| 早期在线日期 | 1月 2023 |
| DOI | |
| 出版状态 | 已出版 - 5月 2024 |
基金资助
No Statement Available
联合国可持续发展目标
此成果有助于实现下列可持续发展目标:
-
可持续发展目标 7 经济适用的清洁能源
关键词
- Transformers
- Computational modeling
- Collaboration
- Servers
- Costs
- Graphics processing units
- Image edge detection
- Collaborative inference
- edge computing
- model decomposition
- vision transformer
收录
- EI
- Scopus
- SCIE
学术指纹
探究 'DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices' 的科研主题。它们共同构成独一无二的学术指纹。引用此
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver