The AI Driving Olympics (AI-DO)

The AI-DO are a set of competitions with the objective of evaluating the state of the art for ML/AI for embodied intelligence
(can AI actually DO anything?).

Overview

Duckietown has been hosting the AI-DO competition finals twice per year, at ICRA (International Conference on Robotics and Automation) and NeurIPS (Neural Information Processing Systems Conference).

AI-DO 2021 finals will take place in conjunction with NeurIPS 2021. There are three leagues:

  • The Urban Driving League uses the Duckietown Platform. Read on to know all about it.
  • The Advanced Perception League uses the nuScenes dataset/challenges and is organized by Motional.
  • The Racing League uses AWS Deepracer and is organized by AWS.  To join the racing league, continuing reading here.

AI-DO 2021 sponsors

Amazon Web Services (AWS)

AI Driving Olympics 2021 @ NeurIPS

The voice of the experts

"It is great to see Duckietown host the AI Driving Olympics at ICRA and NIPS. What a fun way to demonstrate the real challenges in building and deploying self driving cars!"
john leonard
John Leonard, Prof.
Massachusetts Institute of Technology
"Understanding the behavior of AVs is pivotal to assess their riskiness: Swiss Re enthusiastically supports the Duckietown and AI Driving Olympics initiatives."
LDL
Luigi Di Lillo, Dr.
Swiss Re
"The AI Driving Olympics offer a glimpse of the challenges of creating self-driving cars. A playful but rigorous competition on a smaller and safer testbed is the best way to develop the creativity needed to make progress in this field."
Emilio Frazzoli, Prof.
ETH Zurich / Motional
"The AI Driving Olympics is a great way to push the limits of deep learning on physically embodied systems."
Joshua Bengio, Prof.
University of Monreal

Get started with the AI-DO webinars

Introduction to AI-DO 5

ROS and Duckietown baselines

Local development on Duckiebot (DB19)

Reinforcement learning baseline

Imitation learning baseline

Urban Driving League

The urban driving league is based on the Duckietown platform, and includes a series of tasks of increasing complexity aiming at solving precision driving and safety-critical challenges. 

The competition has two stages: participants access the finals by obtaining high placements in the simulated challenges leaderboards, and winners are determined based on experimental evaluations performed in remote Duckietown Autolabs.

The Challenges

The challenges range from single robot tasks such as lane following (LF) on road-loop map to complex multi-robot behaviors such as lane following with intersection and other vehicles in the presence of pedestrians (LFIVP). 

Challenge complexity evolves across several dimensions, the: 

  • road complexity: intersections,  traffic signs or traffic lights,
  • number of vehicles: same or opposite lane, passively or actively controlled
  • presence of pedestrians (duckies) to plan around.

Multi-vehicle challenges support “multiplayer” mode, where your agent is embodied in multiple vehicles.

Each AI-DO edition focuses on a subset of challenges. You can find information on the ongoing AI-DO here, and all the challenges on the challenges server.

Everyone can compete

Participants will not need to be physically present at any stage of the competition.

Competitors submit their solutions to specific challenges in the form of agents packaged as a Docker container.

The agents are evaluated first in simulation (remotely, locally and/or in the cloud), and then the same code is tested on physical robots in a Duckietown Autolab.

The technical infrastructure supporting the AI-DO Urban Driving League is described here.

Resources

We provide tools for competitors to use in the form of simulatorslogscode templatesbaseline implementations and low-cost access to robotic hardware.

Challenge server

The challenges server allows to control one’s submissions, and to see the leaderboards.

Get Started with the AI-Driving Olympics

(Hidden) Webinars

  • Lane Following (LF), in which you need to follow a lane.
  • Lane Following with Pedestrians Vehicles (LFP), in which you need to avoid the duckie-pedestrians.
  • Lane Following with other Vehicles, multibody (LFV_multi), in which your agent is embodied in multiple vehicles.
  • AI-DO 5

    AI-DO 5 (NeurIPS 2020) Webinar 1
    • Introduction to AI-DO 5
    AI-DO 5 (NeurIPS 2020) Webinar 2
    • ROS + Duckietown baseline
    AI-DO 5 (NeurIPS 2020) Webinar 3
    • Local development on Duckiebot (DB19)
    AI-DO 5 (NeurIPS 2020) Webinar 4
    • Reinforcement learning baseline
    AI-DO 5 (NeurIPS 2020) Webinar 5
    • Imitation learning baseline

    AI-DO 3 (2019)

    AI-DO 3 (2019) Webinar 1
    • Introduction to AI-DO 3
    • Minimal agent template
    AI-DO 3 (2019) Webinar 2
    • ROS template 
    • Duckietown baseline
    AI-DO 3 (2019) Webinar 3
    • TensorFlow template
    • Imitation learning from logs and using a simulator
    AI-DO 3 (2019) Webinar 4
    • PyTorch template
    • Reinforcement learning baseline

    AI-DO 3 (2019) Webinar 5

    • Training in the cloud with SageMaker

    AI-DO 3 (2019) Webinar 6

    • Local development: deploying on the Duckiebot (DB18)

    (Hidden) Previous Webinars

    Dates
    Presenter
    Topics
    Recorded Video

    Thurs. Oct 31

    • Introduction
    • Description of the challenges
    • Minimal agent template

    Fri. Nov 1

    • Classic template (ROS)
    • Duckietown baseline

    Mon. Nov 4

    • Tensorflow template
    • IL logs
    • IL simulator

    Tues. Nov. 5

    • Pytorch template
    • RL

    Wed. Nov. 6

    • Training in the cloud

    Thurs. Nov 14

    • Running on the Duckiebot

    FAQs

    Do I need to attend the conference to compete?

    No! If you are not present at the conference where the finals will be hosted we will run your submission on your behalf.

    How do I get help?

    Join the Duckietown international Slack community and ask away!

    How do I get the hardware to test on a real robot?

    Specially crafted hardware kits for each challenge are available here. For any question, you can reach out to [email protected]

    Past Competitions

    The first edition of the AI-DO took place in December 2018, at the 2018 Neural Information Processing Systems (NeurIPS), the premiere machine learning conference, in Montréal. This was the first ever competition with real robots to take place at NeurIPS. AI-DO 1 only had the Urban league, and only the Lane Following challenge. There were over 1600 submissions from 58 unique participants.

    Read a summary of the event here.

    Crowd watching Duckietown

    The second edition of AI-DO took place at the 2019 International Conference on Robotics and Automation (ICRA), with finals held in Montréal, Canada, in May 2019. 

    AI-DO 2 comprised again only the Urban league, but additional challenges were added, such as Lane following with other vehicles (LFV) and Lane following with other Vehicles and Intersections (LFVI). The number of submissions to AI-DO 2 was similar to the number for AI-DO 1.

    Find out who won!

    The third edition was held at NeurIPS 2019 with finals held in Vancouver, Canada. In AI-DO 3 we introduced the advanced perception and racing leagues. AI-DO 3 received over 2000 submissions across all of the leagues.

    Read all about it!

    The fourth edition was scheduled for ICRA 2020 in Paris, France, but was unfortunately cancelled as a result of the COVID-19 outbreak. 

    Duckie overlooking Duckietown
    The fifth edition of AI-DO was in conjunction with NeurIPS 2020. Due to the COVID-19 pandemic, the conference and finals are held virtually. AI-DO 5 will comprise two leagues: Urban Driving and Advanced Perception, with novel challenges in each.

    The sixth edition of AI-DO is in conjunction with NeurIPS 2021. It features three leagues: Urban Driving, Advanced Perception, and Racing. A new challenge is included in the Urban Driving league, and new Duckiebots are used.