SeamsTalk: Seamless Talking Face Generation via Flow-Guided Inpainting

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 8
  • Download : 0
Talking face generation aims to generate a face video to speak according to a given audio or driving video. Despite the importance of natural lower face movement, previous approaches have focused only on animating the lip, neglecting the connection between the modified lower face and the original background. As a result, the generated face is not smoothly integrated into the original video. To address this, we propose a new method to create a seamless talking face video by reformulating talking face generation as a conditional video in-painting. Moreover, since previous methods solely rely on referencing the original frame, the original frame's lip shape influences a generated lip shape. Therefore, we devise a two-stage pipeline that leverages the original frame to reduce scene-specific information loss in the lower face and then utilizes multiple other frames to generate a desired lip shape. Experimental results demonstrate that our method generates a seamless talking face while maintaining similarly accurate lip shapes compared to existing methods.
Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Issue Date
2024-03
Language
English
Citation

IEEE ACCESS, v.12, pp.46678 - 46689

ISSN
2169-3536
URI
http://hdl.handle.net/10203/319826
Appears in Collection
AI-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0