Closed-loop Servoing using Real-time Markerless Arm Tracking

We present a simple, efficient method of realtime articulated arm pose estimation using stochastic gradient descent to correct unmodeled errors in the robot’s kinematics with point cloud data from commercial depth sensors. We show that our method is robust to error in both the robot’s joint encoders and in the extrinsic calibration of the sensor; and that it is both fast and accurate enough to provide realtime performance for autonomous manipulation tasks. The efficiency of our technique allows us to embed it in a closedloop position servoing strategy; which we extensively use to perform manipulation tasks. Our method is generalizable to any articulated robot, including dexterous humanoids and mobile manipulators with multiple kinematic chains.