Shape from Multi-Bounce Light Paths

Classical shape estimation techniques often have simplified but unrealistic assumptions, such as light interacting with the scene with only single-bounce light paths and the scene being Lambertian. However, a real scene often interacts with light in significantly more complex ways; multi-bounce light paths are ubiquitous and real-world materials are typically non-Lambertian. In such cases, shape estimation is not just
challenging, but also beyond the reach of commonly used techniques. We propose a shape estimation framework that deals with multi-bounce light paths. The proposed framework uses light paths, as opposed to images, as the primitive for
shape estimation. The core of our idea is that we can trace the optical journey of a collection of photons as light paths with multiple bounces, where each bounce is an
instance of a light-object interaction. These interactions can often be explained by simple physical laws that govern how properties of light rays change; for example, a
mirror simply changes the orientation of light rays while preserving their radiance; a diffuse wall scatters light in all directions and changes the light ray’s radiance. Shape
estimation problem now reduces to identifying underlying physical phenomena related to each bounce in the light path. Our proposed shape recovery framework is particularly effective for scenes that interact with light in complex ways. We explore three such scenarios. First, we characterize
how information pertaining to the shape of a transparent object is encoded in the deflection of light rays and use it to recover the shape of transparent objects.
Second, we characterize how the path length of two-bounce light paths are sufficient for concave shape recovery. Finally, we show a specialized scenario where the object
of interest can only be imaged through multi-bounce light paths. In all three scenarios, we show that by using physical properties related to each bounce of light paths, we can
find constraints on the geometry of the scene, this leads to accurate shape estimation algorithms. The techniques developed in this thesis are applicable to a wide range of
research fields. The image formation discussed in this thesis shares a lot of similarities in other fields, such as medical imaging, acoustic imaging, and wifi localization.
We hope this thesis can inspire more researchers to deal with multi-bounce effects in
different research fields.