Dissertation Defense

Minimal 3D Priors for Sparse View Reconstruction

Chris RockwellPh.D. Candidate
WHERE:
3725 Beyster Building
SHARE:

Hybrid Event: 3725 BBBZoom  Passcode: 655424

Abstract: Sparse view reconstruction is a fundamental problem in computer vision with applications in extended reality, generative AI and robotics. While learned methods have found great success in 2D and more recently dense view 3D, obtaining 3D from sparse views is an under-constrained problem challenging these models.

The key idea of this dissertation is that minimal 3D priors can enable learned methods to succeed in this difficult case. We apply this to the central tasks of novel view synthesis and camera pose estimation. Our core contributions include fusing a 3D representation with a powerful generative model for novel view synthesis; and fortifying a Vision Transformer with classical geometry, both implicitly and explicitly, for camera pose estimation.

We further show this framework generalizes to the dense-view setting. Our dense-view pose estimation pipeline combines foundation models with a 3D solver, enabling it to handle challenging dynamic scenes and facilitating annotation of the largest diverse, dynamic camera pose dataset.

 

Organizer

CSE Graduate Programs Office

Faculty Host

Prof. David Fouhey and Prof. Justin Johnson