Flexible Perception for High-Performance Robot Navigation

Real-world autonomy requires perception systems that deliver rich, accurate in formation given the task and environment. However, as robots scale to diverse and rapidly evolving settings, maintaining this level of performance becomes increasingly brittle and labor-intensive, requiring significant human engineering and retraining for even small changes in environment and problem definition. To overcome this bottle neck, this thesis advances flexible robot perception by improving its generalizability, adaptivity, and uncertainty-awareness, enabling robots to operate effectively across more environments with minimal additional human intervention.

First, to enable stronger zero-shot generalization, we introduce MapItAnywhere (MIA), a scalable ecosystem for generalizable bird’s-eye view (BEV) mapping. At its core, MIA provides a data engine for automated curation of BEV maps using crowd-sourced, publicly available data. This advances flexible perception by leveraging existing world-scale labels from disparate data sources to improve BEV mapping performance in previously unseen areas, without requiring additional manual data collection, labeling, or curation.

However, even generalizable perception systems face inevitable performance drops when deployed in new environments. To bridge this gap automatically, we develop ALTER, a perception system that adapts online to new environments while mitigating catastrophic forgetting and label noise. ALTER advances flexible perception by intro ducing a system that automatically labels new data using LiDAR and groups them in latent space for efficient retraining, enabling perception systems to operate at higher performance in new scenarios without human intervention

Lastly, while an adaptive perception system can improve over time, collecting data in low-information regions leads to inefficient learning. To this end, we present MapEx, an indoor exploration algorithm that builds an uncertainty aware representation using an ensemble of world model predictors. MapEx advances flexible perception by jointly leveraging map prediction uncertainty and sensor coverage to guide data collection, enabling improved perceptual understanding in new environments without human supervision and reducing the need for manual data collection.

This thesis advances the core capabilities of generalization, adaptation, and uncertainty awareness needed for flexible and automated robot perception. Together, these capabilities address the fundamental bottlenecks of current engineering-intensive workflows and bring us closer to scalable real-world autonomy.