MSc Thesis Presentation - Luis Bolanos

Name: Luis Bolanos
Date: 11th April, 2024
Time: 11:00am PT
Location: ICCS 146 (or Zoom
https://ubc.zoom.us/j/4860834568?pwd=UTNVSEVkSjFjeEdpV0o4bW0rMVVLZz09)
Supervisor: Leonid Sigal, Co-supervisor: Helge Rhodin

Title: Gaussian Shadow Casting for Neural Characters

Abstract:

High-fidelity character models form an integral part in multiple domains, including visual effects, animation, gaming, telepresence, fashion, and more.

Recently, approaches of learning and creating 3D animatable avatars through neural rendering have obtained impressive rendering results, even when learning from monocular video data. These neural characters, while achieving highly detailed geometry and texture reconstruction, bake lighting and shadow information into the model, leading to artifacts when generating novel views, novel poses, or when relighting the model. The highest quality models are trained on data that is uniformly lit, and will otherwise contain artifacts in the shape reconstruction if using illuminated data. Consequently, The use of these neural characters is limited and has not seen widespread adoption in the aforementioned industries.

In this thesis, we address the limitations of neural character lighting, specifically shadow casting, enabling explicit and efficient light calculations while disentangling shadows in the learned texture. Our primary contributions include: 1) A framework for approximating the shape of neural characters using a sum of 3D anisotropic Gaussians; 2) The derivation of a simple analytic formula for integrating the sum of anisotropic Gaussians along a ray; 3) A framework to perform light and shadow calculations using a deferred rendering approach for easy integration on existing models. These contributions enable the training of neural characters from outdoor lit data, and the relighting of the neural characters in a wide array of environments, increasing their utility in industry.