Dependability analysis and hardening of vision transformers against soft errors
Lester Frias-Dominguez, José M. Badía, Germán León, Adrián Amor-Martín, Jose Antonio Belloch
Universidad Carlos III de Madrid Universitat Jaume I
阅读操作
确认中在文库中上传 PDF 后可生成中文音频讲解。
摘要与影响
The deployment of Vision Transformers (ViTs) in safety-critical domains needs a clear understanding of their resilience to soft errors, since their specific layer-level vulnerabilities are currently insufficiently characterized. This work presents a dependability analysis of the ViT-Base architecture against injection-induced soft errors. Using a high-fidelity, software-level fault injection methodology with custom CUDA kernels, the study injects random bit-flips directly into the IEEE 754 binary32 floating-point representation of the intermediate data tensors resulting from the Transformer modules to quantify model accuracy degradation across increasing bit error rates. As a primary result, a vulnerability map across ViT layers is presented, confirming that the results of normalization and fully connected layers exhibit critical sensitivity to soft errors. To address these vulnerabilities, the work evaluates targeted hardening strategies. These include Fault-Aware Training (FAT), applied both globally and selectively to linear layers, as well as practical runtime mitigations such as range-based value clipping and filtering of non-numeric values. The findings demonstrate that these software-only approaches can significantly protect model accuracy.
逐年被引趋势
关键指标
同类平均 = 1
同领域 · 同年份 · 同类型
Google Scholar 与 OpenAlex 的被引统计范围不同,数值存在差异属正常。
AI 辅助阅读
依据:摘要
可就本文提问;依据不足时会说明。
学术脉络
学科主题
工程Radiation Effects in Electronics
Adversarial Robustness in Machine Learning · Distributed systems and fault tolerance