PhD Defence - Weiwei Sun

Name: Weiwei Sun

Date: Friday, September 27, 2024

Time: 9AM - 12PM

Location: Zoom: https://ubc.zoom.us/j/62589054076?pwd=WDRLIawn1tLjbtIm7ASJ2HemNYlOxt.1
Meeting ID: 625 8905 4076
Passcode: 196171

Supervisor’s name: Kwang Moo Yi, Andrea Tagliasacchi

Title of thesis: Exploring Explicit Models for Geometric Point Cloud Learning

Abstract: 

In this thesis we are interested in processing point clouds -- a set of unordered points -- specifically in Euclidean space, such as 3D point cloud acquired from a range sensor (LiDAR) or 4D correspondence cloud in stereo matching task. Point clouds play an increasingly essential role in many tasks due to prevalence they hold. However, it is notoriously challenging to process point clouds with deep neural networks because of their irregular data structure, the difficulty in encoding contextual information from nearby points, and the large compute requirement that is typically required. 

This thesis addresses these challenges by enforcing intermediate features or model parameters to carry specific meanings such as attention and poses, leading to explicit representation. The meanings of explicit representation allow for traditional ways of manipulating features in order to solve target tasks.  We refer to these architectures with explicit representations as explicit models. Explicit models significantly improve performances without massively scaling up training data or model size because the explicit representation directly injects the prior knowledge needed by target tasks into neural networks without any learning.  We explore explicit models for point cloud learning to perform robust estimation, stereo matching, segmentation, reconstruction and neural rendering. The thesis is organized into four chapters: 1,  ACNe: An optimization-inspired network architecture that allows learning with point clouds contaminated with an abundance of outliers. 2, Canonical Capsules: An equivariant latent representation that consists of pose and pose-invariant features, enabling point cloud auto-encoding in unaligned datasets. 3, NeuralBF: A novel 3D instance proposal generation inspired by traditional bilateral filtering for top-down instance segmentation for 3D point clouds. 4, PointNeRF++: A multi-scale, point-based NeRF architecture, allowing seamless integration of point-based representation with Neural Radiance Fields.

Across these four chapters, we show that explicit models significantly improve point cloud learning, inspiring more future research in this domain.

We conclude with a discussion about future works, practical tips on how to form an explicit model, and it's role in the era of large foundation models.