Advances in anytime-valid sequential inference

This thesis contains some advances in the field of “anytime-valid sequential inference”, a  paradigm of statistical inference where confidence intervals, p-values, and hypothesis tests are  valid for all sample sizes simultaneously, including data-dependent stopping times. In more  practical terms, anytime-valid procedures allow an analyst to collect data sequentially over  time and stop sampling for any data-dependent reason without inflating type-I error rates.  

Even in the non-sequential (“batch”) setting, there are two broad categories of statistical  procedures: nonasymptotic and asymptotic ones. Neither is universally preferable to the  other, with nonasymptotic methods enjoying stronger guarantees in finite samples, and with  asymptotic ones being more widely applicable and simpler to implement. This thesis studies  anytime-valid inference in both regimes and is correspondingly divided into two parts.  

The first part focuses on nonasymptotic inference and concentration inequalities. Here,  we introduce new methods for both anytime-valid and batch inference for means of bounded  random variables when sampling with and without replacement. These computationally  and statistically efficient algorithms find several applications in risk-limiting election audits,  off-policy evaluation in contextual bandits, and concentration inequalities under differential  privacy constraints. Each application has a dedicated chapter.  

The second part studies asymptotic anytime-valid inference, a far less mature corner of the  literature. As such, there is an increased focus on articulating the right definitions of anytime valid procedures and their guarantees, as well as laying some of the requisite probabilistic  foundations. In particular, we develop distribution-uniform strong laws of large numbers  and strong Gaussian coupling inequalities which are then used to provide a framework for  asymptotic anytime-valid inference. As one illustrative application of this framework, we  develop the first sequential test of conditional independence that does not rely on the Model-X  assumption.