Striatal Activity Reinforces Decisions Along Emergent Axes

Selecting the appropriate action for a given context is fundamental for the survival of vertebrate animals. Given that environmental contexts are dynamic and often novel, decision-making poses a significant computational task. Not only must an action be selected from a vast range of possible actions, it must be matched to the specific environment and must be selected quickly. The basal ganglia are a network of subcortical nuclei that have long been implicated in decisionmaking. The striatum, the input nucleus of the basal ganglia, has specifically been shown to be of importance, due in part to its integration of a wide variety of inputs into two projection populations. Spiny projection neurons of the direct pathway (dSPNs) or indirect pathway (iSPNs) are traditionally though to select for or against specific actions, respectively. However, alternative interpretations assert that dSPNs and iSPNs produce opposing reinforcing signals, rather than directly select actions. Here, we have used a novel combination of closed-loop optogenetic stimulation and self-directed, ethologically-relevant animal behavior to investigate the question of whether dSPNs and iSPNs reinforce or select behavior, as well as how dSPNs and iSPNs interact within their larger circuit context. 

First, we differentiated between action selection and reinforcement by administering brief optogenetic stimulation of either dSPNs or iSPNs following the detection of self-directed left turns in a Y-maze. To determine whether SPN activity selected the direction of actions, we administered stimulation to SPNs in the right hemisphere, in the left hemisphere, or bilaterally. Analysis of turning behavior showed that dSPNs positively reinforced left turns while iSPNs negatively reinforced left turns. We found this effect in behavioral contexts with and without external rewards, indicating that SPN-mediated reinforcement is not dependent on an external reward. These results demonstrate that the primary projection populations of the striatum reinforce specific behavioral policies rather than directly select behaviors or modulate action value. 

Next, we utilized the behavioral paradigm that we had previously developed to investigate the precise relationship of dSPNs and iSPNs. We administered optogenetic stimulation of both dSPNs and iSPNs concurrently following left turn detection. We found that stimulation of both dSPNs and iSPNs led to negative reinforcement of left turns. This result was consistent regardless of whether turning was motivated by an external reward or not. Interestingly, we found no generalization of negative reinforcement, even without the inclusion of an external reward. These results indicate that dSPNs and iSPNs are not perfectly opponent populations, but instead have a more nuanced relationship that is nevertheless reinforcing. 

We next investigated the role of dopamine (DA) in regulating SPN-mediated reinforcement. DA signaling is necessary for the cortico-SPN plasticity by which SPN-mediated reinforcement occurs. DA signaling is also commonly thought to convey reward information. However, we have found that SPN-mediated reinforcement does not require the inclusion of an external reward. To determine the role of striatal DA release in SPN-mediated reinforcement, we stimulated dopaminergic terminals in dorsal striatum following self-directed left turns. We found that DA release following self-directed left turns did not modulate action value, but instead promoted exploration of the initially non-preferred turn direction. This indicates that striatal DA release acts as a value-neutral salience signal, an important distinction that explains the how SPN-mediated reinforcement can occur outside of explicitly rewarded contexts. 

In the final Chapter, we discuss innovations in the study of ethological animal behavior and its neural correlates, how these innovations can impact the evolution of models of striatal function, and the importance of studying SPNs within their circuit context. Taken together, the results detailed here demonstrate the nuanced interplay between several striatal cell types and how their interactions support striatal reinforcement of animal behavior.