Real time three-dimensional head pose recovery for model-based video coding.

Abstract

This thesis presents an Interactive Facial Animation Testbed (IFAT) for a low bit-rate videophone system based on a face (head) model. The effects of head motion and facial expressions are combined in the 3D head model. The head "pose" (p&barbelow;osition, o&barbelow;rientation, and s&barbelow;cale e&barbelow;stimation) has to be accurately recovered before attempting to recover the face expressions. For the specification of the facial expressions we adopted the "muscle-based" facial model parameterization. This topology independent system incorporates 16 muscles and 10 parameters controlling mouth opening, jaw rotation, eye movement, eyelid opening, and head orientation. Two algorithms were developed for automatic head pose recovery: 2½D pose recovery of the 3D position and 2D orientation using a 2D elliptical head model, and 3D pose recovery of the 3D position and 3D orientation using a 3D wireframe head model. The developed global motion tracking system is meant to work in a realistic videoconferencing environment with no makeup highlighting the speaker's facial features, uncalibrated camera, unknown lighting conditions and unknown scene background. In order to validate the accuracy of the 3D head tracking system, we developed a rapid calibration technique using a sequence of images of a synthetic "standard" 3D head in lieu of real life head.