An Energy- and Resource-Efficient Parallel-Pipelined Pedestrian Detector With Multiscale Image Computation Scheduling for Always-On Intelligent Edge Devices

Histogram of Oriented Gradients (HOG) and linear Support Vector Machine (SVM) have been widely used for pedestrian detection in applications like video surveillance, automatic driving, and intelligent robots. However, in Internet of Things (IoT) applications relying on intelligent edge devices, it is a big challenge to design a high frame-rate multi-scale pedestrian detector without sacrificing precision under strictly resource-limited and energy-constrained conditions. This paper proposes a HOG-SVM-based pedestrian detector with a novel multi-scale image scheduling method based parallel-pipelined multi-detector architecture to maintain a high frame rate with small hardware overhead, and an optimized inter-module pipeline design to minimize pipeline cycles and on-chip buffer costs. Besides, a fine-grained block-score Multiply-Accumulate (MAC) segmentation and mapping method is proposed for the SVM-classifier MAC array to reduce resource overhead while maintaining the same throughput. FPGA validation shows that as compared to the state-of-the-art design with 12 multi-scale detectors, our proposed design achieves a frame rate of 288 fps using only 2 parallel detectors, which reduces LUT, FF, BRAM, and Digital Signal Processor (DSP) usage by 68.9%, 76.1%, 63.3%, and 94.4%, respectively, while improving energy efficiency by 48.6%. ASIC implementation further improves the energy efficiency by 97% and at the same time increases the frame rate to 400 fps at 200 MHz.