Evaluation of Face Detectors and Feature Association Metrics for Real-Time Multi-Face Tracking

Abstract

Video annotation, control of camera direction, labelling and other tasks can benefit from online visual multi-face tracking. Given the availability of high quality general purpose detectors and tracking-by-detection frameworks, we develop a multi-face tracker and comparatively evaluate its components. In this thesis, we train common object detectors on large databases of faces to understand how well these detectors can perform, specifically on faces. We evaluate different face association methods and appearance metrics with different classifier loss functions to track detected faces across frames. We find that while online tracking based on combining state-of-the-art methods can lead to high-quality tracking results, there is still a large gap between offline and online methods. We develop a multi-tracking system in order to achieve an online and real-time standard, one that can track most of the faces in unconstrained settings.