Wide-Baseline Light Fields – Imaging and Applications

The world around us is filled with complex visual phenomena that we may wish to understand or recreate. Iridescence, intricate geometry, and complex spatial texture are all examples of such phenomena that can be challenging to both capture and represent. In order to characterize them, we need to consider how light interacts with different spatial points over a broad range of incoming and outgoing directions. Light field cameras provide a framework for capturing this spatial and angular information in a single image exposure; however, traditional hand-held light field cameras only observe a small fraction of the cone of light emitted by a given scene point. As a consequence, the study of interesting angular effects like iridescence are beyond the scope of such cameras. 

The aim of this thesis is to present a novel light field imaging device capable of measuring over a wide baseline and to explore the space of applications that would significantly benefit from such light field data. We motivate why and how wide-baseline light fields (WBLFs) open up many exciting new capabilities in light field processing, 3D shape reconstruction, and iridescent reflectance capture. 

The core contribution of this thesis is the imaging system design for a WBLF camera. We achieve a wide-baseline by imaging the scene indirectly through an ellipsoidal mirror. The combination of a light field camera and the ellipsoidal mirror provides rich measurements in space and angle. We further analyze the captured light fields to understand the system’s capabilities, including considerations of resolution, angular range, coverage, and depth of field. Additionally, we develop a novel calibration procedure to account for the unique challenges present in such an imaging system. 

We show this WBLF camera in action through a set of di?erent applications that benefit from our data. First, we develop a suite of geometric processing algorithms to provide a parallel to the techniques employed in traditional light field imaging, including viewpoint synthesis, refocusing, and shape reconstruction. In particular, our shape reconstruction technique leverages the unique benefits of the WBLF camera to jointly estimate 3D shape and surface normals. Second, we develop a network similar to state-of-the-art 3D scene representations to train over WBLF data and produce 3D shape and reflectance reconstructions for small objects with varied appearance. Finally, we show that our system is capable of measuring the high dimensionality of the spatially-varying bidirectional reflectance distribution function (SVBRDF) efficiently. We additionally develop a neural SVBRDF representation to render the captured data over a broad range of novel viewpoints, illuminations, and geometries. 

In total, these contributions provide a significant advance to light foeld imaging by establishing the foundation for wide-baseline light field imaging and its applications.