Zixin Yin (殷子欣)

Email: zixin.yin[at]connect.ust.hk       Google Scholar      Github

I am currently a PhD student at the Hong Kong University of Science and Technology under the supervision of Prof. Lionel Ni and Prof. Harry Shum from 2021. In the meantime, I work closely with Baoyuan Wang and Duomin Wang as a research intern at Xiaobing.ai from 2022. My research interests include talking head synthesis, representation learning, disentanglement and 3D face reconstruction.

Before that, I received my B.S. from Department of Engineering Computer Science and Technology (Honors Science Program) and satisfied the requirements of the Honors Youth Program in Xi'an Jiaotong University in 2021.

We introduce a simple and novel framework for one-shot audio-driven talking head generation. Unlike prior works that require additional driving sources for controlled synthesis in a deterministic manner, we instead probabilistically sample all the holistic lip-irrelevant facial motions (i.e. pose, expression, blink, gaze, etc.) to semantically match the input audio while still maintaining both the photo-realism of audio-lip synchronization and the overall naturalness.

We present a novel one-shot talking head synthesis method that achieves disentangled and fine-grained control over lip motion, eye gaze&blink, head pose, and emotional expression.  We represent different motions via disentangled latent representations and leverage an image generator to synthesize talking heads from them.

The website template was adapted from Duomin Wang.