Fault-tolerant Real-Time Perception for Self-Driving Vehicles

Emerging automated vehicle (AV) systems are increasingly deploying various types of perception applications to satisfy

the requirements of safety and convenience. Platform cost and power consumption concerns also drive automotive

system designers to engineer better autonomous systems that share minimum system resources. These new trends lead to many challenges for designing resource sharing and scheduling that provide predictable performance for multiple

heterogeneous applications. In short, perception, resource management and fault-tolerance support have to run concurrently and work together well. Many prior studies have focused on very specific layers, perception alone, or only fault-tolerant needs ignoring the rest of the system. This thesis studies each of these three inter-related subsystems and proposes frameworks to ensure that perception tasks work well together in coordinated, real-time and fault-tolerant

fashion. We analyze the perception needs for AVs and provide practical insight into the real-time resource usage patterns of several software platforms for various applications on an autonomous vehicle. We also introduce detailed methodologies to analyze the computational workloads of the heterogeneous perception tasks. To meet the real-time requirements of AVs, hardware platforms typically include a variety of computing resources ranging from multi-core processors to hardware accelerators such as Graphics-Processing Units (GPUs). We, therefore, introduce novel analytical and systems techniques for running multiple heterogeneous perception applications together. Specifically, we focus on the issues of memory contention, synchronization, and access control for hardware accelerators. In conjunction with analyzable real-time scheduling techniques, we also study the different failure modes of these high-performance computing platforms and develop software strategies to ensure fault-tolerant operations. Our solutions are readily applicable

to commodity hardware not only for migrating existing perception applications to single GPU-based embedded

platforms but also for developing new software and hardware perception systems for AVs.