Basal Ganglia Dynamics in Structure Learning
We frequently encounter new challenges that necessitate action to achieve positive outcomes. To navigate these challenges, our biological brains must utilize environmental feedback from past experiences, combined with knowledge of patterns and structures in the environment, to guide the selection of the most rewarding actions. Central to this process is the basal ganglia, which is crucial for facilitating actions in response to dopaminergic signaling [1] [2] [3]; however, the precise relationships between synaptic plasticity, large-scale network dynamics, and reinforcement learning computations are not fully understood. Additionally, it remains unclear how this system interacts with other brain regions, particularly the hippocampus, which may provide information regarding environmental structure [4] [5] [6] [7].
In this dissertation, I investigate the neural mechanisms underlying reinforcement learning in the basal ganglia and its interaction with structure learning systems. First, I introduce and demonstrate the capabilities of CBGTPy, a framework designed to simulate biologically realistic neural dynamics and plasticity within the cortico-basal ganglia-thalamic (CBGT) system. This flexible framework allows researchers to explore the system’s internal dynamics at multiple scales across various simulated behavioral tasks. Second, I examine the computational value of hippocampal representations in reinforcement learning by employing a model of place cells derived from successor representation [8]. These place cells, along with alternative handcrafted representations, are used to bias cortical inputs within the CBGTPy network throughout its performance of an example structured task [9].
The combined model exhibits enhanced flexibility and improved performance in structured reinforcement learning tasks, mirroring observed human behaviors in similar environments. The extent of this facilitation, however, heavily depends on the properties of the supplied place cells. These findings suggest that hippocampal computations are well-suited for learning task structures, though the current CBGTPy model has limited capacity to fully leverage the information present in biological place cells. This research underscores the importance of multi-region interactions in the brain’s ability to solve structured tasks, offering significant insights into the neural basis of decision-making and learning.
Funding
CRCNS Circuit-Level Mechanisms of Adaptive decision-making
National Institute on Drug Abuse
Find out more...CRCNS: Decision dynamics in cortico-basal ganglia-thalamic networks
National Institute on Drug Abuse
Find out more...History
Date
2024-09-01Degree Type
- Dissertation
Department
- Neuroscience Institute
Degree Name
- Doctor of Philosophy (PhD)