Carnegie Mellon University
Browse
Shi_cmu_0041O_11176.pdf (4.35 MB)

Color Diffuser: Photo-Realistic Image Colorization via Diffusion Feature Extractor

Download (4.35 MB)
thesis
posted on 2024-06-05, 21:33 authored by Zhiyi Shi

 Colorizing images is a challenging problem due to its multi-modal uncertainty and high degree of ill-posedness. Directly training a deep neural network often results in inaccurate semantic colors and a lack of color richness. Although recent methods offer improved outcomes, the capability to extract semantic information and the quality of image generation are still limited. To further improve the results, we propose Color Diffuser, a comprehensive approach to recovering vivid colors by leveraging the rich and diverse color priors encapsulated in the pre-trained diffusion model. Specifically, our method comprises a pixel decoder and a query-based color decoder. The pixel decoder integrates the semantic features obtained from the diffusion inversion process. The color decoder utilizes rich semantic features to refine color queries, thus eliminating the need for manually designed priors. Through cross-attention, our two decoders collaborate to establish correlations between color and multi-scale semantic representations, significantly mitigating the color bleeding effect. Extensive experiments demonstrate that Color Diffuser outperforms existing state-of-the-art methods qualitatively. 

History

Date

2024-05-03

Degree Type

  • Master's Thesis

Department

  • Information Networking Institute

Degree Name

  • Master of Science (MS)

Advisor(s)

Zico Kolter

Usage metrics

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC