Champion-level Drone Racing using Deep Reinforcement Learning (Nature, 2023)

First-person view (FPV) drone racing is a televised sport in which professional competitors pilot high-speed aircraft through a three-dimensional circuit. Each pilot sees the environment from their drone’s perspective via video streamed from an onboard camera. Reaching the level of professional pilots with an autonomous drone is challenging since the robot needs to fly at its physical limits while estimating its speed and location in the circuit exclusively from onboard sensors. Here we introduce Swift, an autonomous system that can race physical vehicles at the level of the human world champions. The system combines deep reinforcement learning in simulation with data collected in the physical world. Swift competed against three human champions, including the world champions of two international leagues, in real-world head-to-head races. Swift won multiple races against each of the human champions and demonstrated the fastest recorded race time. This work represents a milestone for mobile robotics and machine intelligence, which may inspire the deployment of hybrid learning-based solutions in other physical systems. Reference: Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun, Davide Scaramuzza Champion-Level Drone Racing using Deep Reinforcement Learning. Nature, August 30th, 2023 DOI: 10.1038/s41586-023-06419-4 PDF: https://www.nature.com/articles/s4158... To see more of our work on drone flight and machine learning, check out our webpage: Drone racing: https://rpg.ifi.uzh.ch/research_drone... Deep learning: https://rpg.ifi.uzh.ch/research_learn... Agile flight: https://rpg.ifi.uzh.ch/aggressive_fli... Lab's publications: https://rpg.ifi.uzh.ch/publications.html Affiliation: Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, and Davide Scaramuzza are with the Robotics and Perception Group, University of Zurich, Switzerland: https://rpg.ifi.uzh.ch/ Matthias Müller and Vladlen Koltun are with Intel Labs.