Carnegie Mellon University
Browse

Basal Ganglia Dynamics in Structure Learning

Download (16.82 MB)
thesis
posted on 2024-10-23, 20:31 authored by Matthew ClappMatthew Clapp

 We frequently encounter new challenges that necessitate action to achieve positive  outcomes. To navigate these challenges, our biological brains must utilize environmental  feedback from past experiences, combined with knowledge of patterns and structures in  the environment, to guide the selection of the most rewarding actions. Central to this  process is the basal ganglia, which is crucial for facilitating actions in response to  dopaminergic signaling [1] [2] [3]; however, the precise relationships between synaptic  plasticity, large-scale network dynamics, and reinforcement learning computations are  not fully understood. Additionally, it remains unclear how this system interacts with  other brain regions, particularly the hippocampus, which may provide information  regarding environmental structure [4] [5] [6] [7].  

In this dissertation, I investigate the neural mechanisms underlying reinforcement  learning in the basal ganglia and its interaction with structure learning systems. First, I  introduce and demonstrate the capabilities of CBGTPy, a framework designed to  simulate biologically realistic neural dynamics and plasticity within the cortico-basal  ganglia-thalamic (CBGT) system. This flexible framework allows researchers to explore  the system’s internal dynamics at multiple scales across various simulated behavioral  tasks. Second, I examine the computational value of hippocampal representations in  reinforcement learning by employing a model of place cells derived from successor  representation [8]. These place cells, along with alternative handcrafted representations,  are used to bias cortical inputs within the CBGTPy network throughout its  performance of an example structured task [9].  

The combined model exhibits enhanced flexibility and improved performance in  structured reinforcement learning tasks, mirroring observed human behaviors in similar  environments. The extent of this facilitation, however, heavily depends on the  properties of the supplied place cells. These findings suggest that hippocampal  computations are well-suited for learning task structures, though the current CBGTPy  model has limited capacity to fully leverage the information present in biological place  cells. This research underscores the importance of multi-region interactions in the  brain’s ability to solve structured tasks, offering significant insights into the neural basis  of decision-making and learning. 

Funding

CRCNS Circuit-Level Mechanisms of Adaptive decision-making

National Institute on Drug Abuse

Find out more...

CRCNS: Decision dynamics in cortico-basal ganglia-thalamic networks

National Institute on Drug Abuse

Find out more...

History

Date

2024-09-01

Degree Type

  • Dissertation

Department

  • Neuroscience Institute

Degree Name

  • Doctor of Philosophy (PhD)

Advisor(s)

Timothy Verstynen

Usage metrics

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC