3D Art Research with Radiance Field and Style Transfer

Architects and designers desire to put real-life scenes in the digital world to assist their work, such as experimenting with textures, meshes, and lighting conditions. Artists may create prototypes of art pieces based on digital scenes. The above practices mutually require 3D-style transfer technology. However, existing 3D reconstruction method such as photogrammetry suffers from slow computation, defects for large scenes with sparse inputs, and lacks support to content-aware postprocessing. Existing style transfer mainly focuses on the 2D image style transfer, which does not work well in the 3D scene. This thesis aims at a lightweight and simple method to solve problems and meet practical needs. It takes Neural Radiance Field(NeRF) as a backbone and combines the 2D style transfer with the photo-realistic 3D scene from NeRF to achieve 3D style transfer. In methodology, it discusses the generation of the dataset and different approaches to combine NeRF and Style transfer network, optimization to improve the results. Besides, this thesis shows a valid comparison of different approaches from both analytic and aesthetic aspects. It further explores the applications that can utilize such techniques to achieve architecture design and art creation in a fashion of 3D-aware style control.