Attention distillation for low-cost driver behavior recognition

Deploying artificial intelligence (AI) models for driver behavior recognition has become increasingly popular for improving driving safety. However, as vehicle intelligence advances, the rising number of AI models imposes heavy demands on onboard hardware. This study aims to develop low-cost driver behavior recognition models to alleviate such burdens, where “low-cost” refers to smaller model size and faster inference speed. To this end, we construct shallower convolutional neural networks (CNNs) by removing deeper convolutional layers and propose an innovative Attention Distillation (AD) mechanism to enhance their performance. Compared to previous distillation mechanisms, attention distillation provides more detailed knowledge transfer: it leverages the category-specific attention of deeper CNNs as the teacher to guide the attention of shallower CNNs. Experimental results show that MobileNetV2-11AD, which retains only 11 residual bottlenecks and is trained with AD, achieves higher accuracy than the original MobileNetV2 on the State Farm and AUC V2 datasets, while reducing parameters by 89.29 % and boosting the frame rate on CPU and GPU by 57.52 % and 72.65 %, respectively. Consistent gains are observed on the diverse 100-Driver dataset. These results confirm that the proposed AD mechanism provides a promising solution for low-cost driver behavior recognition. The code is available at https://github.com/gaohangcodes/AD4DBR .