This thesis studies algorithms for teaching autonomous agents to complete tasks through trial and error learning. Typically, this problem is posed as a reinforcement learning (RL) problem, wherein agents attempt to maximize a user-provided reward function. The algorithms studied here take a different approach, largely eschewing the reward function and instead learning to achieve desired outcomes directly from data. This approach allows users to employ algorithmic tools from the supervised and unsupervised learning, while also surfacing an interface that allows non-expert users to teach agents new tasks....