Active Robot Perception using Programmable Light Curtains

Most 3D sensors used in robot perception, such as LiDARs, passively scan the entire environment while being decoupled from the perception system that processes the sensor data. In contrast, active perception is an alternative paradigm for robotics where a controllable sensor adaptively focuses its sensing capacity only on the most useful regions of the environment. Programmable light curtains are a recently-invented, resource-efficient, active sensor that measure the depth of any user-specified surface (“curtain”) at a significantly higher resolution than LiDAR. The main research challenge is to design perception algorithms that decide where to place the light curtain at each timestep, tightly coupling sensing and control in a closed loop. 

This thesis lays the algorithmic foundations for active robot perception using programmable light curtains. We investigate the use of light curtains for various perception tasks such as 3D object detection, depth estimation, obstacle detection and avoidance, and velocity estimation. First, we incorporate the velocity and acceleration constraints of the light curtain into a constraint graph; this allows us to compute feasible light curtains which optimize any task-specific objective. Then, we develop a suite of algorithms using a variety of tools such as Bayesian inference, deep learning, information gain and dynamic programming to intelligently place light curtains in the scene. 

Finally, we combine multiple intelligent placement strategies in an online learning framework. First, we are able to explicitly estimate velocities and positions of scene points using a Bayes filtering technique based on particle filters and occupancy grids. Then, we propose a novel self-supervised reward function that evaluates the accuracy of current velocity estimates using future light curtain placements. This insight enables an online multi-armed bandit framework to intelligently switch between multiple placement policies in real time, outperforming individual policies. These algorithms pave the way for controllable light curtains to accurately, efficiently and purposefully perceive complex and dynamic environments.