Data and Humans in Algorithmic Risk Assessment

Algorithmic risk assessment instruments (RAIs) are increasingly used to aid decision makers in high-stakes domains. Characterizing the risks and opportunities of RAIs requires understanding how the data, the design of the tools, their evaluation, and their use affect the resulting decisions. In this thesis, we focus on two components of this design pipeline: the role of selection biases in the data (Part I) and of the interactions between RAIs and human decision makers (Part II). 

Part I of the thesis covers analyses of the selection biases in criminal justice data that stem from the discrepancy between offense and arrest. We propose sensitivity analyses for fairness evaluations of recidivism RAIs when not all offenses result in arrests. Then, we present three studies that estimate racial disparities in the likelihood of arrest for a criminal offenses The first study focuses on disparities in police enforcement for marijuana violations by comparing data from police records with estimates of underlying criminal behavior obtained from survey data. The second study employs police records of violent crimes. The third study extends this analysis by introducing methods to adjust arrest rates computed on police records for unreported offenses, using victimization data. 

Part II of the thesis discusses analyses of the influence of RAIs on human judgment. We first investigate how the deployment of an RAI affected decisions in child maltreatment hotline screenings. In particular, we take advantage of a glitch in the implementation of the tool to study whether decision makers adhere to the algorithmic  acommendations when the risk is misestimated. Then, we present the results of two vignette studies involving predictive tasks that crowdworkers perform with the aid of RAIs. In the first study, we analyze the ecological validity of various study design choices. In the second study, we test whether homophily between humans and RAI, as well as monetary incentives, affect the behavior of the participants. We conclude with a user study that assesses the impact of human-RAI workflow configurations on the decision-making of expert veterinary radiologists in a clinical imaging task.