Learning by Experimentation: The Operator Refinement Method

Autonomous systems require the ability to plan effective courses of action under potentially uncertain or unpredictable contingencies.  Planning requires knowledge of the environment that is accurate enough to allow reasoning about actions.  If the environment is too complex or very dynamic, goal-driven learning with reactive feedback becomes a necessity. This chapter addresses the issue of learning by experimentation as an integral component of PRODIGY. PRODIGYis a flexible planning system that encodes its domain knowledge as declarative operators, and applies theoperator refinement methodto acquire additional preconditions or postconditions when observed consequences diverge from internal expectations.  When multiple explanations for the observed divergence are consistent with the existing domain knowledge, experiments to discriminate among these explanations are generated. The experimentation process isolates the deficient operator and inserts the discriminant condition or unforeseen side-effect to avoid similar impasses in future planning.  Thus, experimentation is demand-driven and exploits both the internal state of the planner and any external feedback received.  A detailed example of integrated experiment formulation in presented as the basis for a systematic approach to extending an incomplete domain theory or correcting a potentially inaccurate one.