Carnegie Mellon University
Browse
- No file added yet -

Interactive Machine Learning from Humans: Knowledge Sharing via Mutual Feedback

Download (11.4 MB)
thesis
posted on 2023-10-20, 19:11 authored by Pallavi KoppolPallavi Koppol

People regularly interact with human-in-the-loop learning (HiLL) agents that attempt to adapt to their priorities, tastes, and preferences. Examples of such systems include web search engines, movie recommender systems, text prediction, and even large-language model based chat applications. To be adaptive, these HiLL systems must first learn an accurate model of an individual’s behavior and preferences. 

The ability to learn such a model depends on the quality of the information the HiLL system is able to elicit from the people with whom it interacts, and how well it is then able to leverage that information. Typically, this information is generated via a loop where an agent or model takes actions or makes suggestions that a person responds to with some feedback, and that response is then used to train future behavior. Henceforth, we will refer to this query-feedback pair as an interaction. 

We note that the informativeness of a learning interaction is limited by how fully it empowers a person to share their knowledge. We demonstrate that it is possible to improve a person’s teaching performance by providing them with (1) more appropriate modalities for sharing feedback (i.e. interaction types) and (2) insight into the context of the learner they are instructing. Our approach therefore moves towards a model that prioritizes a human teacher’s ability to provide informative feedback. 

To do this, we first formalize the space of interactions that can be used to learn from human feedback and present four interaction archetypes: Showing, Categorizing, Sorting, and Evaluating. Then, we analyze the effects that these different interaction types may have on learning outcomes via both direct and indirect influences on collected training data. We build on this to contribute a learning approach that enables an algorithmic learner to learn from multiple interaction types based on which would be the most immediately informative. Finally, we develop and evaluate an interaction type-based approach towards bridging the gap between an algorithmic learner and a human teacher’s mental model of that learner. 

History

Date

2023-09-29

Degree Type

  • Dissertation

Department

  • Computer Science

Degree Name

  • Doctor of Philosophy (PhD)

Advisor(s)

Reid Simmons, Henny Admoni

Usage metrics

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC