Real time detection of driver fatigue based on CNN‐LSTM

Fatigue driving is one of the main causes of traffic accidents. In order to solve this problem, a new Convolutional Neural Network and Long Short‐Term Memory (CNN‐LSTM) based real‐time driver fatigue detection method is proposed. First of all, using simple linear clustering algorithm (SLIC), the driver's image is divided into super pixels of uniform size, which are used as input of CNN, and CNN is trained to automatically learn the features of eyes and mouth contained in the image, and then the location and area of eyes and mouth are obtained by using the trained CNN. On this basis, the eye feature parameter Perclos, mouth feature parameter MClosed and face orientation feature parameter Phdown are extracted, and the above feature parameters on the continuous time series and steering wheel angle feature parameter SA are taken as the input of LSTM, and the fatigue level is taken as the output to detect the fatigue state of the driver in real time. Experimental data shows that this method can not only overcome the influence of illumination, background, angle and individual differences, but also the accuracy of detection can reach 99.78%, and the average detection time is 16.94 ms/frame.