A Decision Science Approach Towards Improving the Study of Adherence Information

Imperfect adherence to research protocols is a ubiquitous problem across studies evaluating the effectiveness of interventions. Imperfect adherence decreases the average outcome effects measured during clinical trials and observational studies. Many methods exist to measure adherence which differ in their accuracy and costs, depending on the type of intervention. Study designs improve adherence through reminders to participants or evaluation during a run-in period. Analysis methods can account for imperfect adherence, but they vary in terms of the statistical assumptions required and the potential bias that each introduces. 

This dissertation applies decision science principals to create recommendations addressing how adherence should be measured, analyzed, and reported for future studies. Decision scientists can fill many roles in terms of informing policy analysis. Three roles addressed by this dissertation include (a) translating human behavior into analytic terms, (b) treating analysis as human behavior, and (c) facilitating two-way communication between experts and lay people. 

Chapter two addresses (a) by using a simulated clinical trial of a common hypertension drug in order to inform when collecting adherence information could be beneficial to researchers. The trial evaluates two trial designs, a randomized controlled trial with and without an enrichment period, and two analysis methods, intent-to-treat analysis and per-protocol analysis. This simulated trial finds that the costs of using a less pragmatic study design or more biased analysis method does not outweigh the benefits of the monetary savings that these approaches provide.

Chapter three addresses (b) by evaluating how imperfect adherence is accounted for by researchers of software as medical device (SaMD) apps. The chapter uses normative criteria from the literature to evaluate how researchers measure and analyze adherence information. Our review finds that the majority of trials imperfectly reported adherence metrics (63%) and did not use an appropriate method to evaluate efficacy (75%). Future SaMD trials could be improved by reporting all facets of adherence, preregistering efficacy analyses, and using less biased analysis methods. 

Chapter four continues to address (b) by applying the recommendations found from Chapter three to evaluate the MyHealthyPregnancy (MHP) app. The MHP app collected observational data between September 2019 and February 2022. Metrics of each facet of usage find that usage of the MHP app during the study was very low. For example, the majority of participants used an app activity less than three days throughout the trial. An evaluation of the average treatment effect finds no evidence that the MHP app is effective at increasing gestational age at birth or reducing preterm birth risk. Analyzing the dose-response effect for app usage finds that increased use is correlated with increased gestational age at birth. The interpretation of this finding is limited by our knowledge of confounders and low usage of the app by study participants. In the future, studies could be improved through a randomized trial design and increased app usage. 

The sum of this work has identified many gaps between how researchers are currently accounting for imperfect adherence and how they could address this source of bias in the future. Researchers would benefit from measuring adherence both in terms of how users interact with an intervention and how they interact with recommendations from the intervention. They would also benefit from measuring and reporting on all facets of adherence. Analysis of adherence could be improved through the preregistration of efficacy analyses and more careful consideration of the confounders that could affect these analyses. Regulators could support these changes by creating clearer guidelines for accounting for imperfect adherence and more strict requirements for the preregistration of observational studies.