Mengyi Shan

I am a third year PhD student at the Paul G Allen School of Computer Science in University of Washington where I am affiliated to the Graphics and Imaging Lab (GRAIL) and the Reality Lab. I am co-advised by Steve Seitz, Brian Curless and Ira Kemelmacher-Shlizerman.

I studied at Harvey Mudd College in Claremont, California with double majors in Computer Science and Mathematics and a concentration in linguistics. I was advised by TJ Tsai in the Music Information Retrieval Lab.

Before college, I grew up in Beijing China. I spent seven years at the High School Affiliated to Renmin Univeristy of China (RDFZ), in the first class of Early Development Program (EDP).

Email  /  CV  /  Google Scholar  /  Twitter  /  Github

Research

I work on generating creative contents with modern technologies in vision and graphics.

We combine the 2D photorealism and 3D controllability in human video generation by conditioning video diffusion models on controlled rendering of 3D avatars.

We build datasets with multi-person pose and motions, and jointly train to generate natural and diverse group motions of multiple humans from textual descriptions.

We generate diverse and realistic animal motion sequences from textual descriptions, without a large-scale animal text-motion dataset.

We present a system that automatically brings street view imagery to life by populating it with naturally behaving, animated pedestrians and vehicles.

We introduce a high resolution, 3D-consistent image and shape generation technique trained on single-view RGB data only, and stands on the shoulders of StyleGAN2 for image generation.

We study the problem of automatically generating piano score following videos given an audio recording and raw sheet music images.

Misc

When I am not coding or reading papers, I enjoy math competition puzzles, detective fictions, board games, Guzheng (Chinese zither), and kpop idols.