Learning to Predict and Make Decisions under Distribution Shift

A common use case of machine learning in real world settings is to learn a model from historical data and then deploy the model on future unseen examples. When the data distribution for the future examples differs from the historical data distribution, machine learning techniques that depend precariously on the i.i.d. assumption tend to fail. So dealing with distribution shift is an important challenge when developing machine learning techniques for practical use. While it is unrealistic to expect a learned model to predict accurately under any form of distribution shift, well chosen research objectives may still lead to effective machine learning algorithms that handle distribution shift properly. For example, when facing distribution shift, we may expect to build machine learning models that (i) make accurate predictions under specific assumptions on how the distribution shift happens; (ii) identify out-of-distribution inputs where the model may not be able to predict well; and/or (iii) act conservatively according to what it can predict well. While recent research has produced practically successful methods in machine learning settings with independent and identically distributed data, progress on settings where dealing with distribution shift is necessary has remained in a comparatively developmental stage.

In this thesis, we study the problem of learning under distribution shift in two scenarios: prediction and decision making. The first part of the thesis addresses the prediction problem, focusing on exploiting specific assumptions such as covariate shift and label shift. We develop theoretical understanding and effective techniques in these scenarios. In the second part of this thesis, we study the problem of offline policy optimization, where the goal is to learn a good policy from a fixed set of data, whose distribution may not be rich enough to inform the optimal policy. We first present an extensive empirical study on behavior regularized offline reinforcement learning algorithms. We then present a theoretical study on whether/why one should follow the pessimistic principle in the offline policy optimization problem.