Video-Based Facial Re-Animation

Generating photorealistic facial animations is still a challenging task in computer graphics, and synthetically generated facial animations often do not meet the visual quality of captured video sequences. Video sequences on the other hand need to be captured prior to the animation stage and do not offer the same animation flexibility as computer graphics models. We present an inexpensive method for video-based facial animation, which combines the photorealism of real videos with the flexibility of CGI-based animation by extracting dynamic texture sequences from existing multi-view footage. To synthesize new facial performances, these texture sequences are concatenated in a motion-graph-like way. In order to ensure realistic appearance, we combine a warp-based optimization scheme with a modified cross dissolve to prevent visual artefacts during the transition between texture sequences. Our approach makes photorealistic facial re-animation from existing video footage possible, which is especially useful in applications like video editing or the animation of digital characters.