Gesture-Based Programming for Robotics: Human Augmented Software Adaptation

Gesture-Based Programming is a paradigm for programming robots by human demonstration in which the human demonstrator directs the self-adaptation of executable software. The goal is to provide a more natural environment for the user as programmer and to generate more complete and successful programs by focusing ontask experts rather thanprogramming experts. We call the paradigm “gesture-based” because we try to enable the system to capture, in real-time, theintention behind the demonstrator’s fleeting, context-dependent hand motions, contact conditions, finger poses, and even cryptic utterances in order to reconfigure itself. The system is self-adaptive in the sense that knowledge of previously acquired skills (sensorimotor expertise) is retained by the system and this knowledge facilitates the interpretation of the gestures during training and then provides feedback control during run-time.