Efficient Convolutional Neural Networks for Low-power Automotive Processors

Abstract

The last decade has seen a significant expansion of the application of ConvNet models to many important tasks, especially for applications where it is necessary to perceive and categorize the surrounding environment as scanned by sensors. This includes areas of application such as drones, mobile devices, and especially autonomous driving. In autonomous driving, the detection of vulnerable road users such as cyclists and pedestrians is critical for the safe operation of vehicles. These vehicles may be given a degree of control over their driving inputs. Additionally, detecting other road vehicles is essential for the safe operation of vehicle features. The availability of increasingly capable devices (especially GPUs) for developing such models has meant that power limits and computational complexity are less of a concern than outright detection capability. However with compute-restricted platforms, this becomes an issue since restrictive power limits or computational capability place tight restraints on the allowable ConvNet models that can be employed. We introduce two models that look at pedestrian detection in 2D, and car and cyclist detection in 3D, running on embedded platforms with restrictive power constraints. We show how they can be optimized to their respective task and outperform competing models while being significantly faster. In the case of the 3D model, we show how competing approaches on the same model misidentify the weaknesses of the model, that we leverage as strengths. Our improvements allow the model to consume point clouds in real-time with higher detection performance and a far faster rate, on an embedded platform, than directly competing models. Our work does not preclude low-level optimizations such as precision calibration (quantization), layer fusion or memory optimization. We then extend the well known KITTI dataset by hand, providing fine semantic segmentations for its Car class while using the data to perform an analysis of an existing LiDAR and image fusion model. This analysis shows how the choice of feature fusion for our target architecture can drastically alter model performance. We leverage these insights to propose a further modification to PPslim called PPslimg. This updated model fuses ground plane estimations and exceeds the performance of a competing approach for the same model, that relies on fusing computationally expensive semantic segmentation features.