Semi-Autonomous Manipulation of Natural Objects

Effective deployment of robots in search and rescue missions will enable faster and safer removal of debris and other hazardous material. As a result, additional lives will be saved, the number and severity of injuries will decrease, and significant damage to infrastructure will be avoided. Already today robots are an integral part of many search and rescue units. These robots typically serve as either mobile cameras (autonomy in navigation, but no manipulation capabilities) or as tools controlled by a human operator (no autonomy, but capable of interacting with the environment). We propose a robotic manipulator that shares a role with the human operator: the robot provides the operator with processed visual information and a set of possible actions, and the human operator chooses the desired next interaction with the environment. To that end, we develop a novel scene segmentation algorithm based on 3D data and a toolbox of compliant motion controllers. We evaluate our approach in real-world experiments in which our robot is tasked with clearing piles of unknown natural objects.