Full-surround 3D Reconstruction using Kaleidoscopes

 3D scanning of a single view of an object seldom suffices. Be it for 3D printing, augmented reality, or virtual reality, scanning of the shape of the entire object in all its complexity—what we refer to as full-surround 3D—is critical to have a faithful digital twin.

 A key factor in achieving full-surround 3D scan is the diversity and number of viewpoints. Standard techniques, involving multiple cameras and sometimes projectors, often become prohibitively expensive and complex, placing them beyond the reach of the average consumer of 3D technology. This thesis proposes a novel and accessible solution by using a kaleidoscope system. Comprising multiple planar mirrors, the kaleidoscope allows for a combinatorial increase in viewpoints through repeated light reflections, all captured by a single camera. This approach enables the construction of a virtual multiview imaging system that is easy to build, calibrate and deploy, with components that are easily available.

 In this thesis, we establish the theoretical and practical foundations of kaleidoscopic techniques for full-surround 3D reconstruction. We are particularly interested in the reconstruction of highly complex objects with intricate geometry that include self-occlusions. This work aims to create a kaleidoscopic equivalent to the multiple view geometry in classical computer vision, making the following contributions.

 First, we explore kaleidoscope design and its calibration. It encompasses the development of crucial design factors, including the number of mirrors and their configuration. Additionally, we formulate metrics for assessing various kaleidoscope designs, enabling the quantitative evaluation of their coverage and accuracy. We also address the precise calibration of the extrinsic parameters of mirrors, a vital aspect of kaleidoscopic reconstruction. This precision is essential, as errors can be significantly amplified through the multiple reflections within the kaleidoscope. 

Second, we introduce kaleidoscopic structured light, which serves as a kaleidoscopic equivalent to traditional structured light. The kaleidoscope generates the structured light system with hundreds of projectors and cameras, providing high accuracy and coverage, thanks to redistribution of the rays diverse directions corresponding to large baselines. It enables the reconstruction of intricate geometry, that include severe self-occlusions, concavities, and large genus number. 

Third, we present kaleidoscopic neural rendering, an adaptation of neural rendering within a kaleido?scopic context. While neural rendering has recently achieved significant advancements in speed and accuracy, its handling of reflections remains inadequate. Our framework addresses this by integrating multiple specular reflections within the kaleidoscope alongside neural surface representations. This integration enables the creation of silhouette-consistent and photo-consistent 3D shapes from a single kaleidoscopic image. This method facilitates single-shot full-surround 3D reconstruction, particularly advantageous for dynamic objects, as a single frame in this setup contains all necessary information for complete surround coverage.