Hybrid Searched Binary Convolutional Neural Network for Human Activity Recognition Using Thermal Videos

Human activity recognition is a quickly evolving approach that identifies the physical activities of human beings from visual or sensor data. Still, human activity recognition using visible images impacts the image quality due to varying lighting conditions. Hence, the thermal video is used for solving such issues. However, the recognition becomes complex due to noisy data and depends on handcrafted features. To tackle such issues, this article develops the novel model named Searched Binary Convolutional Neural Network (SeBCoN) for human activity recognition. The frames are extracted from the input video, and the brightness of the frames is enhanced by the Local Gamma Transform. The human segmentation is performed by the Lovasz–Softmax loss function-based PyramNet, and the pose representation is done using a multi-scale convolution fusion deep residual network (Mscf-ResNet). Finally, the developed SeBCoN recognizes the human activities into direction, eating, discussion, greeting, posing, phone talk, purchasing, smoking, sitting, taking a photo, walking, waiting, running, duckwalk, and crawl. Moreover, the developed SeBCoN-based human activity recognition attained the optimal accuracy, sensitivity, specificity, F1 score, false omission rate (FOR), and Matthews correlation coefficient (MCC) of 96.37%, 95.36%, 97.14%, 96.36%, 4.513%, and 0.935.