Carnegie Mellon University
Browse

Computational Exploration of Higher Visual Selectivity in the Human Brain

Download (62.62 MB)
thesis
posted on 2024-11-14, 19:00 authored by Andrew LuoAndrew Luo

  A fundamental goal of cognitive neuroscience has been understanding how the  human visual cortex supports perceiving and interpreting visual information in the  world around us. Traditional approaches to mapping the visual cortex have relied  on manually assembled stimulus sets, often employing isolated objects in artificial  contexts with simplified backgrounds. These approaches do not fully capture the  complexity and richness of real-world visual experience, potentially biasing results  and limiting our understanding of visual processing. My thesis introduces a suite  of computational approaches leveraging naturalistic image stimuli to identify and  characterize the high-level organization of visual information in the human brain.  Specifically, I present:  

  1. Brain Diffusion for Visual Exploration (BrainDiVE): A method utilizing gradient guidance from a differentiable image-to-fMRI encoder and a pre-trained  image diffusion model to generate naturalistic “most-exciting-inputs” that maximally activate specific brain regions.  
  2.  Semantic Captioning Using Brain Alignments (BrainSCUBA): A technique  unifying the embedding spaces of CLIP image and text embeddings with fMRI  encoder weights to drive a vision-language model. This enables the generation  of natural language descriptions of voxel-wise selectivity within the visual  cortex.  
  3.  Semantic Attribution and Image Localization (BrainSAIL): An approach employing vision foundation models and dense semantic features to localize activating objects within complex naturalistic images across higher-level visual  areas.  

These computational methods are complemented by human validation experiments  using synthetically generated stimuli. Overall, my thesis work demonstrates the power of combining naturalistic stimuli with advanced computational techniques to reveal  the fine-grained organization of the human visual cortex. In addition to providing  a detailed overview of the computational models I have developed, I outline future  computational and fMRI experiments designed to further validate and extend these  findings. My research paves the way towards a more comprehensive and ecologically  valid understanding of visual processing, with implications for building more accurate  models of the brain and contributing to novel applications in artificial intelligence. 

History

Date

2024-10-01

Degree Type

  • Dissertation

Department

  • Neuroscience Institute

Degree Name

  • Doctor of Philosophy (PhD)

Advisor(s)

Michael J Tarr Leila Wehbe

Usage metrics

    Categories

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC