Self-adjusting facial animation retargeting

Abstract

While facial animation capturing focuses on accurate reconstruction of actor's performance, facial animation retargeting has the goal to transfer the animation to another character so that the semantical meaning of the animation remains. Because of the popularity of blendshape animation, this effectively means to compute suitable weights for the given character. Current approaches produce reasonable results when facial proportions of actor and target are similar, which is not the case if stylized characters for games or movies should be animated using facial capturing. Alternatively, methods exists that learn a mapping from examples of matching expressions to guarantee semantic equivalence. Our approach can reliably retarget facial animations from a real actor to a stylized character without any additional examples. In detail we present a new algorithm for automatic blendshape adjustment that is based on the insight that facial motion retain a high level of similarity even for characters of different proportions. Given a sequence of facial motion for training and correspondences for the markers our method is capable, drawing inspiration from Graph-based and Manifold Alignment techniques, to adjust the blendshapes of our target character without any further user interaction. In addition, we incorporate a simple, yet elegant facial prior based on discrete differential properties to guarantee smooth mesh deformation.