Data-Driven Robotic Grasping in the Wild

<div>Robotic grasping has seen tremendous advancements in recent years. Yet, the current paradigm of manipulation research is typically some form of table-top manipulation</div><div>in constrained setups or in simulation. Building general purpose personal robots that can autonomously grasp unknown objects in unstructured environments like homes is</div><div>an open problem. In this thesis, we explore important directions in scaling data-driven grasping to the diversity and constraints imposed by the real world. We first discuss how we can go beyond picking individual objects in isolation to 6-</div><div>DOF grasping in clutter. Most existing methods train policies on datasets collected in curated settings (in lab or simulation) and hence may not cope with the mismatch</div><div>in data distribution when deployed in the wild. We build and open-source a low-cost mobile manipulator platform to parallelize data collection in challenging settings like</div><div>homes and show that policies trained on this data generalize to novel objects in unseen homes. As a result, we also discuss ideas for scaling robot learning with several robots</div><div>and transferring policies between different hardware. Yet, we hypothesize that visual perception alone is insufficient for robustness and present a self-supervised tactile-based</div><div>re-grasping framework to close the loop on grasp execution. Lastly, we strive to go beyond robotic pick-and-place and generalize to diverse semantic manipulation tasks.</div><div>We do so by scaling task-oriented grasping datasets with crowdsourcing and learning from semantic information like knowledge graphs.</div>