Interactive Gaze Driven Animation of the Eye Region

Proceedings of the 21st International Conference on 3D Web Technology (Web3D '16)

  Debanga R. Neog, Joao L. Cardoso , Anurag Ranjan, Dinesh K. Pai 

Sensorimotor Systems Laboratory, University of British Columbia

Best Paper Award

We describe a complete pipeline to model skin deformation around the eyes as a function of gaze and expression parameters. The face is rendered with our WebGL application in real-time and runs in most modern browsers (a) and mobile devices (b). An example of automatic facial animation generation while watching a hockey video is shown (c-d). The character starts watching the match with a neutral expression (c) and gets concerned when a goal is scored (d). Eye movements were generated automatically using salient points computed from the hockey video.

Abstract We propose a system for real-time animation of eyes that can be interactively controlled in a WebGL enabled device using a small number of animation parameters, including gaze. These animation parameters can be obtained using traditional keyframed animation curves, measured from an actor’s performance using off-the-shelf eye tracking methods, or estimated from the scene observed by the character, using behavioral models of human vision. We present a model of eye movement, that includes not only movement of the globes, but also of the eyelids and other soft tissues in the eye region. The model includes formation of expression wrinkles in soft tissues. To our knowlwdge this is the first system for real-time animation of soft tissue movement around the eyes based on gaze input.
Paper PDF (10M)
Video Movie (93M)
Bibtex @inproceedings{Neog:2016:IGD:2945292.2945298,
author = {Neog, Debanga R. and Cardoso, Jo\~{a}o L. and Ranjan, Anurag and Pai, Dinesh K.},
title = {Interactive Gaze Driven Animation of the Eye Region},
booktitle = {Proceedings of the 21st International Conference on Web3D Technology},
series = {Web3D '16},
year = {2016},
isbn = {978-1-4503-4428-9},
location = {Anaheim, California},
pages = {51--59},
numpages = {9},
url = {},
doi = {10.1145/2945292.2945298},
acmid = {2945298},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {WebGL, eyes, gaze, human animation, soft tissue},
Web App Click the following links to run the app:   Desktop App    Mobile App
Funding This work was funded in part by grants from the Canada Research Chairs Program, NSERC, MITACS, CFI, and BC KDF.