Talking face generation by disentangled audio-visual representation and implicit natural code분리된 오디오 비주얼 표현과 함축적인 내추럴 코드에 의한 말하는 얼굴 생성

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 126
  • Download : 0
This paper addresses making a video where a speaker of a video is changed to another identity to look natural. One related research, called Disentangled Audio-Visual System(DAVS), got good results based on the way it disentangles audio-visual representation from images. However, DAVS generates static results that only move the mouth area. Based on DAVS, I aim to make dynamic results with head pose and facial expression. I approach this issue by adding implicit natural code. It is an expanded idea of Zhou et al., called Pose Controllable Audio-Visual System(PCAVS), that controls the pose of talking faces with only 12 dimensions vector, not by 2D or 3D landmark, to implicit natural code. In addition, I add a discriminator for accurate lip sync. With my model armed with the aforementioned components, I conduct comparative experiments with existing talking face generation models using a large-scale audio-visual dataset. In the case of videos with little movement, my model generates visual quality results equivalent to or better than other models. Also, ablation studies for a natural code and a discriminator for lip sync are conducted. It shows that each component does its job well.
Advisors
최성희researcher
Description
한국과학기술원 :전산학부,
Publisher
한국과학기술원
Issue Date
2024
Identifier
325007
Language
eng
Description

학위논문(석사) - 한국과학기술원 : 전산학부, 2024.2,[iv, 19 p. :]

Keywords

말하는 얼굴 생성▼a립싱크▼a이미지로부터 오디오와 비주얼 표현 분리하기▼a함축적인 내추럴 코드; Talking face generation▼aLip-sync▼aDisentangling audio-visual representation from images▼aImplicit natural code

URI
http://hdl.handle.net/10203/321682
Link
http://library.kaist.ac.kr/search/detail/view.do?bibCtrlNo=1097262&flag=dissertation
Appears in Collection
CS-Theses_Master(석사논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0