Carnegie Mellon University
Browse

Distributed Reinforcement Learning for Autonomous Driving

Download (1.67 MB)
thesis
posted on 2023-02-07, 19:35 authored by Zhe Huang

Due to the complex and safety-critical nature of autonomous driving, recent works typically test their ideas on simulators designed for the very purpose of advancing self-driving research. Despite the convenience of modeling autonomous driving as a trajectory optimization problem, few of these methods resort to online reinforcement learning (RL) to address challenging driving scenarios. This is mainly because classic online RL algorithms are originally designed for toy problems such as Atari games, which are solvable within hours. In contrast, it may take weeks or months to get satisfactory results on self-driving tasks using these online RL methods as a consequence of the time-consuming simulation and the difficulty of the problem itself. Thus, a promising online RL pipeline for autonomous driving should be efficiency driven. 

In this thesis, we investigate the inefficiency of directly applying generic single-agent or distributed RL algorithms to CARLA self-driving pipelines due to the expensive simulation cost. We propose two asynchronous distributed RL methods, Multi-Parallel SAC (off-policy) and Multi-Parallel PPO (on-policy), dedicated to accelerating the online RL training on the CARLA simulator via a specialized distributed framework that establishes both inter-process and intra-process parallelization. We demonstrate that our distributed multi-agent RL algorithms achieve stateof-the-art performances on various CARLA self-driving tasks in much shorter and reasonable time. 

Funding

CMU Argo AI Center for Autonomous Vehicle Research

History

Date

2023-01-22

Degree Type

  • Master's Thesis

Department

  • Robotics Institute

Degree Name

  • Master of Science (MS)

Advisor(s)

Jeff Schneider

Usage metrics

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC