Dynamic decision policy reconfiguration under outcome uncertainty

Humans and other mammals flexibly select actions under noisy and unstable conditions. To shed light on the mechanism driving this flexibility, we evaluated how the underlying decision policy evolves when humans change their minds about the most rewarding action. Participants performed a dynamic variant of the two-armed bandit task that manipulated the certainty in relative reward probabilities (conflict) and the reliability of action-outcome contingencies (volatility). We found that conflict and volatility contributed to shifts in exploratory states by changing both the rate of evidence accumulation (drift rate) and the amount of evidence needed to make a decision (boundary height). Following a switch in the optimal choice, the drift rate and the boundary height reduce, allowing variability in the accumulation process to predominate action selection, leading to a fast exploratory state. These changes facilitate the discovery of the new optimal choice, with a quick recovery of the boundary height to baseline. In parallel, the drift rate gradually returns to its asymptotic value as the belief in the value of the optimal choice stabilizes. Together, these decision dynamics suggest that, in the context of volatile two-choice decisions, humans adopt a combined information-threshold and drift rate mechanism in response to environmental changes. Unlike previous observations, we found no evidence that fluctuations in norepinephrine, as measured by pupillometry, associated with this adaptive shift toward an exploratory policy. We conclude that the multifaceted processes underlying a decision can rapidly reconfigure to adapt action selection policy under multiple forms of environmental uncertainty.