Generated by All in One SEO Pro v5.0.0.1, this is an llms-full.txt file, used by LLMs to index the site.
# Duckietown
Learning robotics and AI like the professionals
## Posts
### [Rome Cup 2026](https://duckietown.com/robotics-rome-cup-2026/)
**Published:** May 19, 2026
**Author:** Duckietown Admin
**Excerpt:** Learn more about the Rome Cup 2026, organized since 2007 by the Fondazione Mondo Digitale in Rome. Dedicated to students, businesses and institutions.
**Content:**
# Rome Cup 2026
Learn more about the Rome Cup 2026, organized since 2007 by the Fondazione Mondo Digitale in Rome. Dedicated to students, businesses and institutions.
**Rome, 28th April 2026** – Over 4000 students took part in the 19th edition of the Rome Cup 2026 entitled “*What’s Next? – Intelligence and talent in dialogue, converging technologies and shared governance*” was held in Rome from 28-30 April and organized by Fondazione Mondo Digitale in collaboration with the University of Rome “La Sapienza”.
- [ Rome Cup 2026 ](https://www.romecup.org/)
- [ Fondazione Mondo Digitale ](https://www.mondodigitale.org/eventi/romecup-2026)
- [ La Sapienza University ](https://www.uniroma1.it/it/pagina-strutturale/home)
## Rome Cup 2026
The Rome Cup 2026 is a multi day event dedicated to robotics and
innovation across three key areas – robotics, artificial intelligence
and life sciences – and with a strategic vision: focusing on the younger generations.
Organized by [Fondazione Mondo Digitale](https://www.mondodigitale.org/) and the [University of Rome “La Sapienza”](https://www.uniroma1.it/it/pagina-strutturale/home), the event brings together schools, universities, research centers and companies to discuss about robotics, artificial intelligence and emerging technologies.
Now in its 19th year, the initiative features conferences, robotics competitions, workshops and career guidance sessions, with the aim of nurturing talent and skills to build a sustainable and inclusive future.
[ ![Logo Università "La Sapienza"](https://duckietown.com/wp-content/uploads/2026/05/LoghiSito2-300x150-1.avif "Logo Università "La Sapienza" - Duckietown - Duckietown") ](https://www.uniroma1.it/it/pagina-strutturale/home)
[  ](https://www.mondodigitale.org/eventi/romecup-2026)
Since 2007, Rome Cup has been encouraging the younger generation to
study scientific subjects and developing skills and professional
profiles for the job market. Each edition introduces new “themes” (women
in science, robotics spin-offs, Industry 4.0, life sciences, etc.) to
forge connections and enrich the innovation ecosystem through
vertical and cross-sector partnerships.
The event saw the participation of over 4000 students, involving 100 teams in the robotics competitions, 32 teams in creative contests, 11 universities, 17 university careers talks and an exhibition area featuring prototypes from 53 organizations.
The central theme of this edition was augmented intelligence as a paradigm for human-centered, inclusive and sustainable development of technological innovation and its application ecosystems.
### Duckietown at the Rome Cup 2026
Duckietown, represented by Jacopo Tani, Ph.D., was part of a panel of judges tasked with assigning an award to the best project participating in the robotics creative contest. The “20°Trofeo Internazionale di robotica Città di Roma \[20th International Robotics Trophy, City of Rome\]” (Rescue Line, Explorer Junior, Explorer Senior, Robotic arms) took place as well.
Prototype robotic applications were presented (assistance,
agriculture, etc.) in various categories: AgroBOT, CoBOT, DroneBOT,
MareBOT, NonniBOT, TirBOT, and this year also a HealthBOT category.
Following a brief presentation session, the jury composed by Ezia Palmeri, senior official at the Ministry of Education and Merit; Fabrizio Corradi, psychotechnologist and expert in assistive technologies, augmentative and alternative communication, and artificial intelligence at LUMSA and the Leonarda Vaccari Institute; Alessia Lo Bosco, Director of Vocational Training Services for the Metropolitan City of Rome; Massimiliano Dibitonto, Head of Product and Services Guidelines at Olivetti and Jacopo Tani, co-founder, Chairman and Chief Executive Officer (CEO) of Duckietown, selected the winners.
The winning teams from the 2026 edition:
MAREBOT – Thalassa Boat, IIS Marconi Pieralisi
TIRBOT – Road Safety and AI, IIS De Santis
NONNIBOT – Word Shield, IIS Giordano
AGROBOT – Diet Bot, IIS Russell (Cles)
COBOT – Aura, IIS Avogadro
RobotCT – IIS Vaccarini (Catani
Photo credits: Francesco Vignali
### Learn more about Duckietown
[Duckietown](https://duckietown.com/) is a set of tools that enables hands-on robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
**Categories:** Events
**Tags:** duckietown, robotics, Rome Cup
---
### [Duckie Day 2026 brings robotics to salmon farming](https://duckietown.com/duckie-day-2026-salmon-farming/)
**Published:** July 23, 2026
**Author:** Duckietown Admin
**Excerpt:** Students at Duoc UC Puerto Montt, Chile, used Duckietown to solve a real salmon farming challenge during the 2026 edition of the Duckie Day.
**Content:**
# Duckie Day 2026 brings robotics to salmon farming
Students at Duoc UC Puerto Montt, Chile, used Duckietown to solve a real salmon farming challenge during the 2026 edition of the Duckie Day.
**DUOC UC Puerto Montt, Chile, July 2026:** Duckie Day 2026 brought together students from six Duoc UC campuses at Puerto Montt for a robotics competition focused on one of Chile’s main industries: salmon farming.
##### Quick links
- [ DUOC UC ](https://www.duoc.cl/)
- [ Duckie Day 2026 ](https://www.youtube.com/watch?v=N4s7R3p8qF4)
- [ MOWI Chile ](https://mowi.com/fo/contact/mowi-chile/)
- [ School of Computer Science and Telecommunications, DUOC UC ](https://www.duoc.cl/escuela/informatica-telecomunicaciones/)
## Learning autonomous robotics through industry challenges
Hosted by the [School of Computer Science and Telecommunications](https://www.duoc.cl/escuela/informatica-telecomunicaciones/) at Duoc UC Puerto Montt, Duckie Day 2026 brought together students from six Duoc UC campuses at Puerto Montt to tackle robotics challenges inspired by practical issues present in the aquaculture industry of Chile.
Aquaculture is a major economic activity in Chile. Among the diverse aquacultures practiced in the country, Atlantic salmon aquaculture is by far the largest sector. Chile is the second largest salmon and trout producer in the world after Norway.
### Duckietown, the platform behind the technical learning
Working with the Duckietown platform, teams developed autonomous Duckiebots capable of navigating independently, detecting sea lice using embedded computer vision, issuing an alert, and autonomously returning to their starting point.
Along the way, participants applied core robotics concepts including camera calibration, lane following, AprilTag localization, PID control, and sim-to-real development on ROS-based robots powered Duckietown.
The **main challenge centered on pathogen detection**, a critical concern for the regional industry.
[Luis Tamariz](https://www.linkedin.com/in/luis-t-salamanca/), a participating student, explained the details of the mission: “It was a competition where we had to program robots with cameras that drive autonomously. The main challenge was to go from point A to different points to complete the goal: detect sea lice, issue an alert, and return to the starting point.”
> It was a competition where we had to program robots with cameras that drive autonomously. The main challenge was to go from point A to different points to complete the goal: detect sea lice, issue an alert, and return to the starting point.
>
> Luis Tamariz
[Carolina Martínez](https://www.linkedin.com/in/carolina-martinez-b%C3%B3rquez-77a184145/), Program Director of the School of Computer Science and Telecommunications, highlighted the scale of the gathering:
“We have six campuses participating, a wonderful experience for our students, for our faculty, and for the companies involved. We are proud that our students can use their knowledge of artificial intelligence to demonstrate their connection to the region; these are not just abstract ideas, but a real exercise rooted in the aquaculture industry.
The event was backed by key industry players such as [MOWI](https://mowi.com/fo/contact/mowi-chile/) Chile, who presented real challenges from their processing plants.
Nicolás Mihovilovic, MOWI’s Deputy Manager of Public Affairs, praised the level achieved: “We are very proud of this young talent. These students were able to solve real industry challenges, particularly from Mowi’s processing plant. It took weeks of hard work, planning, and programming.”
### Learn more about Duckietown
[Duckietown](https://duckietown.com/) enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
**Categories:** Events
**Tags:** AI, computer vision, Duckie Day, duckietown, robot autonomy, salmon farming
---
### [Teaching robot autonomy at The Hague University](https://duckietown.com/teaching-robot-autonomy-the-hague-university/)
**Published:** May 19, 2026
**Author:** Duckietown Admin
**Excerpt:** Lecturer Vikram Radhakrishnan shares how he is teaching robot autonomy at The Hague University through hands-on robotics, computer vision, SLAM, and real-world student projects.
**Content:**
# Teaching robot autonomy at The Hague University
Lecturer Vikram Radhakrishnan shares how he is teaching robot autonomy at The Hague University through hands-on robotics, computer vision, SLAM, and real-world student projects.
**The Hague University of Applied Sciences, The Netherlands, May 2026:** Vikram Radhakrishnan is a Lecturer in the Applied Data Science & AI program at The Hague University of Applied Sciences, where he developed a specialization in self-driving vehicles using Duckietown as the primary educational platform.
In this interview, he shares how the course came to life, how students approach robot autonomy and autonomous driving, and what he has learned from introducing hands-on robotics into an applied AI curriculum.
##### Quick links
- [ Vikram Radhakrishnan ](https://www.linkedin.com/in/vikram-mark-radhakrishnan-90038660/)
- [ The Hague University of Applied Sciences ](https://www.dehaagsehogeschool.nl/)
- [ Applied Data Science & AI Program ](https://www.dehaagsehogeschool.nl/opleidingen/hbo-bachelor/applied-data-science-artificial-intelligence)
- [ Self Driving Challenge 2026 ](https://www.selfdrivingchallenge.nl/)
## Teaching robot autonomy with Duckietown
##### Hi, thank you for sharing your time with us! Could you introduce yourself?
My name is Vikram Radhakrishnan, and I am a Lecturer at The Hague University of Applied Sciences, where I teach in the [Applied Data Science & AI program](https://www.dehaagsehogeschool.nl/opleidingen/hbo-bachelor/applied-data-science-artificial-intelligence). It’s a relatively new four-year undergraduate program, and this year we’re graduating our first cohort of students.
In the third year, students can choose to specialize either in Generative AI or in Self-Driving Vehicles, which is the specialization I developed.
##### Could you tell us more about this program?
The course is designed to introduce students to the main building blocks of autonomous driving. We begin with computer vision and image processing, covering topics such as lane following and object detection.
From there we move on to localization techniques, including Simultaneous Localization and Mapping (SLAM), before finishing with navigation, control, and path planning.
The goal is to expose students to the complete autonomous driving pipeline through practical, hands-on exercises.
##### What kind of projects do students complete during the course?
The course contains three major projects.
The first project focuses on lane following combined with object detection, giving students practical experience with computer vision.
The second project is considerably more advanced. Students learn concepts such as Extended Kalman Filter localization and monocular SLAM, including ORB-SLAM3, before selecting an approach to map their environment while driving through the Duckietown.
The final project combines everything they have learned. Students navigate autonomously from one point to another while simultaneously building a map of the environment.
Throughout the course I ask students to record videos of their work and share them with the class, which has been a great way to document their progress.
> Students enjoy working with real robots, the practical nature of the course keeps them engaged because they can immediately see the results of what they implement.
>
> Vikram Radhakrishnan,
> Lecturer in applied data science and A.I., The Hague University of Applied Sciences
##### How are students responding to the course?
They enjoy working with real robots.
The practical nature of the course keeps them engaged because they can immediately see the results of what they implement.
The biggest challenge isn’t motivation but complexity. Some topics, particularly localization and SLAM, involve mathematics that can be demanding for students at a university of applied sciences.
I usually focus on giving them a strong conceptual understanding while allowing them to work with existing implementations rather than diving deeply into every mathematical derivation.
##### Why did you choose Duckietown?
Before developing the course, I looked for a platform that could provide both the hardware and the software needed to teach autonomous driving effectively.
I saw that Duckietown was already being used by multiple universities and research institutes, and what immediately stood out was the amount of educational material that was already available. Having an integrated platform with existing learning experiences made it a very attractive choice.
Initially we purchased three Duckiebots because this was an elective course and we didn’t know how many students would enroll.
Eventually eighteen students signed up, so we rented three additional robots. Today we have six purchased Duckiebots together with three rental units, which allows students to work directly with the hardware.
##### You were among the first instructors to use the Duckiebot rental program. How was your experience?
Overall it has been a positive experience.
We had some delays at the beginning due to shipping and customs, so it took a little longer before the rental robots reached us. Once they arrived, though, having additional robots made a significant difference because all eighteen students needed access to physical hardware.
Looking ahead, we would probably purchase additional robots instead of renting simply because it simplifies logistics.
##### What do you see as the strengths of Duckietown and what were the challenges you encountered?
The biggest advantage is the complete ecosystem.
The hardware, software stack, documentation and educational materials are all available, allowing instructors to build a course without starting from scratch.
For example, the existing lane-following implementation provides an excellent foundation that students can immediately experiment with and extend.
Regarding the challenges, Duckietown provides excellent learning experiences that explain individual components, whether that is ROS, object detection or another topic. However, understanding how all those individual pieces fit together into one complete autonomous system takes time.
Even for instructors, there is a significant learning curve before the overall architecture becomes clear.
I also found that some tutorials and template repositories differ slightly in their structure, which can be confusing for students when they begin developing their own ROS nodes.
Also, most students use Windows laptops, and we initially tried to use development containers, but we weren’t able to get that workflow running reliably. In the end, we asked students to install Linux as a dual-boot.
Making the development environment easier to access across operating systems would certainly lower the barrier for new users.
##### Your students also participate in the Self-Driving Challenge. Is that part of the course?
The competition is actually separate from the course itself.
Our university has participated in the [Self-Driving Challenge](https://www.selfdrivingchallenge.nl/) for several years, and this year the event has become international, with teams from multiple countries taking part.
I think there’s a great opportunity for closer collaboration between the competition, universities teaching autonomous driving, and Duckietown itself.
We’re all working toward the same objective of helping students learn robot autonomy through hands-on experience.
##### Finally, would you recommend Duckietown to other instructors?
Absolutely.
If you’re looking for a practical platform to teach autonomous driving, Duckietown provides a complete educational ecosystem. Students don’t just learn isolated algorithms; they see how perception, localization, planning and control all come together on a real autonomous robot.
That combination of theory and hands-on implementation is what makes the learning experience so valuable.
### Learn more about Duckietown
[Duckietown](https://duckietown.com/) enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
**Categories:** People
**Tags:** AI, AI robotics, autonomous systems, robotics education, university robotics
---
### [Real-Time Reinforcement Learning in Duckiematrix](https://duckietown.com/real-time-reinforcement-learning-in-duckiematrix/)
**Published:** May 7, 2026
**Author:** Duckietown Admin
**Excerpt:** This project explores Reinforcement Learning in Duckiematrix within Duckietown, analyzing real-time delays and their impact on autonomous driving performance.
**Content:**
# Real-Time Reinforcement Learning in Duckiematrix
##### Project Resources
- **Objective**: Evaluate how computation delays affect Reinforcement Learning performance in Duckiematrix autonomous driving tasks.
- **Approach**: Simulate real-time delays in Duckietown and compare classical RL with action-conditioned Real-Time RL policies.
- **Authors**: Guillaume Gagné-Labelle, Gabriel Sasseville and Nicolas Bosteels
[ Mila ](https://mila.quebec/en)
[ Authors ](#authors)
[ Report ](https://gabrielsasseville01.github.io/Duckietown-Real-Time-RL/)
[ Code ](https://github.com/GabrielSasseville01/Duckietown-Real-Time-RL)
## Reinforcement Learning in Duckiematrix Real-Time - objectives and approach
The objective of this project is to evaluate Reinforcement Learning performance in [Duckiematrix](https://docs.duckietown.com/ente/devmanual-duckiematrix/intro.html "Duckiematrix") under real-time constraints in Duckietown by quantifying the impact of computation delay on policy performance, reward, and episode length in autonomous driving tasks.
The approach implements [Soft Actor-Critic (SAC)](https://arxiv.org/abs/1801.01290 "Soft Actor-Critic (SAC)") based Reinforcement Learning models in Duckiematrix simulation, introduces controlled fixed and variable time delays in the environment loop, and compares classical Reinforcement Learning policies π(at | st) with action-conditioned Real-Time Reinforcement Learning policies π(at | st−1, at−1) using evaluation reward, reward variance, and episode length metrics across multiple delay distributions.
[ Learn about autonomous driving in Duckietown ](https://duckietown.com/educational-resources/#machine-learning)
## Reinforcement Learning in Duckiematrix Real-Time - highlights
![Reinforcement Learning sampling time distribution in Duckiematrix SAC model showing inference latency variability]()
Figure 1: Reinforcement Learning SAC Sampling Time Distribution
![Reinforcement Learning reward vs delay in Duckiematrix showing performance degradation]()
Figure 2: Reward vs Delay Classical RL
![Reinforcement Learning training curves in Duckiematrix under delay conditions]()
Figure 3: Learning Curves Classical RL
![Reinforcement Learning episode length vs delay in Duckiematrix simulation]()
Figure 4: Episode Length vs Delay - Classical RL
![Reinforcement Learning performance distribution in Duckiematrix with delay variability]()
Figure 5: Performance Distributions - Classical RL
![Reinforcement Learning loss convergence in Duckiematrix classical RL training]()
Figure 6: Loss Convergence - Classical RL
![Reinforcement Learning reward vs delay in Duckiematrix real-time RL with action conditioning]()
Figure 7: Reward vs Delay - Real-Time RL Action Conditioning
![Reinforcement Learning learning curves in Duckiematrix real-time RL training]()
Figure 8: Learning Curves - Real-Time RL Action Conditioning
![Reinforcement Learning episode length vs delay in Duckiematrix real-time RL]()
Figure 9: Episode Length vs Delay - Real-Time RL
![Reinforcement Learning performance distribution in Duckiematrix real-time RL]()
Figure 10: Performance Distributions - Real-Time RL
![Reinforcement Learning loss convergence in Duckiematrix real-time RL training]()
Figure 11: Loss Convergence - Real-Time RL
![Reinforcement Learning reward comparison in Duckiematrix classical vs real-time RL]()
Figure 12: Side-by-side comparison of reward vs delay for both approaches
![Reinforcement Learning episode length comparison in Duckiematrix across RL methods]()
Figure 13: Episode length comparison
![Reinforcement Learning training curves comparison in Duckiematrix classical vs real-time]()
Figure 14: Training progress comparison
![Reinforcement Learning distribution comparison in Duckiematrix classical vs real-time RL]()
Figure 15: Performance distribution comparison
![Reinforcement Learning loss comparison in Duckiematrix classical vs real-time RL]()
Figure 16: Loss convergence patterns
![Reinforcement Learning performance vs mean delay in Duckiematrix variable delay experiment]()
Figure 17: Performance vs Mean Delay lines = delay regime
![Reinforcement Learning performance delta vs baseline in Duckiematrix delay variability]()
Figure 18: Absolute Difference vs Fixed Baseline lines = delay regime
## The challenges
The challenges in this project involve modeling Reinforcement Learning in Duckiematrix under real-time constraints in Duckietown where computation delay violates the Markov Decision Process assumption of instantaneous state–action transitions, resulting in state–action mismatch, policy instability, and reward degradation. Fixed and variable delay distributions introduce non-stationarity, increased variance in evaluation reward, and failure modes at higher delays (≥0.1s for classical RL and ≥1.0s for both methods), while stochastic latency from neural network inference impacts policy execution timing, convergence behavior, and sample efficiency.
Additional challenges include maintaining stability in Soft Actor-Critic training under delayed feedback, handling missing training metrics, ensuring robustness across delay distributions, and evaluating performance using consistent metrics such as reward, variance, and episode length across multiple experimental conditions.
##### Looking for similar projects?
Check out the following works on sim-to-real with Duckietown:
- [Reproducible and Low-cost Sim-to-Real Environment for Traffic Signal Control](https://duckietown.com/reproducible-sim-to-real-traffic-signal-control-environment/)
- [Simulation to Real Domain Adaptation for Lane Segmentation](https://duckietown.com/sim2real-lane-segmentation-via-domain-adaptation/)
- [Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Real ](https://duckietown.com/sim2real-transfer-of-multi-agent-policies-for-self-driving/)[](https://duckietown.com/deep-reinforcement-learning-for-autonomous-lane-following/)
## Real-Time Reinforcement Learning in Duckiematrix: authors
[  ](https://www.linkedin.com/in/gabrielsasseville/)
[Gabriel Sasseville](https://www.linkedin.com/in/gabrielsasseville "Gabriel Sasseville") is a Ph.D. student at [Mila](https://mila.quebec/en "Mila") Institute in Montreal, Canada.
[  ](https://mila.quebec/en/directory/guillaume-gagne-labelle)
[Guillaume Gagné-Labelle](https://www.linkedin.com/in/guillaume-gagn%C3%A9-labelle-99603827b/ "Guillaume Gagné-Labelle") is a collaborating researcher at [Mila](https://mila.quebec/en "Mila") Institute in Montreal, Canada.
[  ](https://mila.quebec/en/directory/nicolas-bosteels)
[Nicolas Bosteels](https://mila.quebec/en/directory/nicolas-bosteels "Nicolas Bosteels") is a Master’s Research student at [Mila](https://mila.quebec/en "Mila") Institute in Montreal, Canada.
### Learn more
Duckietown is a modular, customizable platform for robotics and artificial intelligence education, enabling hands-on learning and real-world experimentation with autonomous systems.
Designed for teaching, learning, and research, Duckietown supports the full spectrum of autonomy development, from foundational computer science and robotics concepts to advanced AI and self-driving systems research.
These spotlight projects are shared to demonstrate how Duckietown bridges theory and practice in robotics and AI, empowering students to apply machine learning and autonomy techniques to physical robots while building practical skills valued in academic research and industry.
[ Get Started ](https://duckietown.com/guides/)
[ More Projects ](https://duckietown.com/best-robotics-and-ai-projects-for-learning-autonomy/)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Control & Reinforcement Learning, Post categories, Projects
**Tags:** perception, project, sensor fusion, student project
---
### [Duckiebots now available pre-assembled and initialized](https://duckietown.com/duckiebot-db21j-now-available-pre-assembled-and-initialized/)
**Published:** February 5, 2026
**Author:** Federico Tani
**Excerpt:** Save time and start using your robot right out of the box with pre-assembled and pre-initialized Duckiebots, now available on the Duckietown shop.
**Content:**
# Duckiebots now available pre-assembled and initialized
Save time and start using your robot right out of the box with pre-assembled and pre-initialized Duckiebots, now available on the Duckietown shop.
Thanks to the feedback of instructors in the Duckietown community, Duckiebots (from DB21J v3) are now available pre-assembled and initialized. Customized names are available, too.
Assembling the hardware and initializing the software of your Duckiebot can take up to more than six hours, assuming you have an already functional working environment. Doing it once is a great learning experience; repeating the process for an entire class may become tiresome.
##### Quick links
- [ Learn about Duckiebot assembly and initialization services ](https://get.duckietown.com/collections/duckiebot-services)
- [ Get a pre-assembled and pre-initialized Duckiebot ](https://get.duckietown.com/products/duckiebot-db21?variant=45917225451695)
## Why pre-assembly and initialization services?
Duckiebot kits require **assembly**, **initialization**, and **calibration** to operate autonomously.
- **Assembly** is about putting together the mechanical part of the robot and making sure each component works according to specifications.
- **Initialization** is about installing the software and setting it up correctly.
- **Calibration** is about fine-tuning the behavior of sensors and actuators, for example, performing camera intrinsics and extrinsics calibrations.
While calibrations are very sensitive and should be performed just before using the Duckiebot in its actual operating environment, assembly and initialization are time-consuming activities that are fun to perform a few times, but not repeatedly.
An **experienced** Duckietowner might take 2-3 hours to assemble a Duckiebot, roughly one hour to initialize it, and another hour to test, address potential mistakes, update, and overall make sure that everything is working as it should. And this is assuming having already a correctly set up working environment, i.e., workstation and network.
For an inexperienced Duckietown user, it will take longer than 4-5 hours to get everything up and running.
This is why **we now offer the option to acquire Duckiebots that are directly assembled, initialized, and up-to-date** before they reach you.
## What do you get with a pre-assembled and initialized Duckiebot
The Duckiebot pre-assembly and initialization service ensures that:
- All hardware components are recognized and operating nominally upon receiving your Duckiebot
- Your Duckiebot ships with some residual battery charge (subject to shipping regulations)
- Firmware on the HUT and battery are up to date with the latest version
- Duckiebot Software is initialized with the latest *ente* release
- Network is pre-installed, and the credentials are provided
- Your Duckiebot(s) is provided with a standard name (e.g., duckiebot01, duckiebot02, …, duckiebotNN)
- You get a video of the quality control tests done on each of your Duckiebots.
## What should you do after receiving a pre assemble and initialized Duckiebot?
After receiving your Duckiebot, there will be a few last steps to take care of before being able to drive autonomously down your Duckietown. For example, you will:
- need a Duckietown unless you already have one;
- have to remove the protective covers from the camera, time-of-flight sensor, and screen
- customize the network credentials to your own (or, create a default duckietown network)
- calibrate the camera and odometry of your Duckiebot, and fine tune autonomy pipeline parameters for your specific environment.
[ Get a pre-assembled & initialized Duckiebot ](https://get.duckietown.com/products/duckiebot-db21?variant=45917225451695)
[ Check out all Duckiebot Services ](https://get.duckietown.com/collections/duckiebot-services)
### Learn more about Duckietown
[Duckietown](https://duckietown.com/) enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
**Categories:** News
**Tags:** DB21Jv3, duckiebot, Duckietown services, pre-assembled, pre-assembly, pre-initialization, pre-initialized
---
### [Sim-to-Sim-to-Real Transfer for Small Autonomous Vehicles](https://duckietown.com/sim-to-sim-to-real-transfer-for-small-autonomous-vehicles/)
**Published:** December 12, 2025
**Author:** Duckietown Admin
**Excerpt:** This project studies Sim-to-Real Transfer in Duckietown using high- and low-fidelity simulators to predict autonomous vehicles performance on Duckiebots.
**Content:**
# Sim-to-Sim-to-Real Transfer for Small Autonomous Vehicles
##### Project Resources
- **Objective**: To evaluate Sim-to-Real Transfer accuracy by comparing learning and control performance across simulators of varying fidelity.
- **Approach**: Train, validate, and benchmark identical perception and control pipelines across low-fidelity simulation, high-fidelity simulation.
- **Authors**: Jurriaan Buitenweg, Dr. Cynthia Liem
[ TU Delft ](https://www.tudelft.nl/en/)
[ Authors ](#authors)
[ Thesis ](https://repository.tudelft.nl/record/uuid:07d3a6cf-1ca9-429f-b51b-6ab17637524c)
[ Report ](#report)
## Sim-to-Real Transfer for Small Autonomous Vehicles - objectives and approach
This study investigated whether an intermediate, low-fidelity simulator can be used to estimate the simulation-to-reality (Sim2Real) gap for autonomous driving models trained in a high-fidelity simulator.
Specifically, it proposes a Sim-to-Sim-to-Real evaluation pipeline in which deep reinforcement learning models are trained in [CARLA](https://carla.org/), evaluated in [Gym Duckietown](https://docs.duckietown.com/daffy/devmanual-software/intermediate/simulation/gym-simulation-in-duckietown.html), and finally deployed on a physical [Duckiebot](https://get.duckietown.com/).
The objective is to determine to what extent performance in Gym Duckietown predicts real-world performance, and whether similarity in learned feature representations can serve as an indicator of successful Sim-to-Real transfer before deployment in the real world.
[ Learn about autonomous driving in Duckietown ](https://duckietown.com/educational-resources/#machine-learning)
## Sim-to-Real Transfer for Small Autonomous Vehicles - highlights
![Sim-to-Real reinforcement learning loop for autonomous vehicles showing agent, environment, state, action, and reward interaction]()
Reinforcement Learning Interaction Loop
![Sim-to-Real experimental setup for autonomous vehicles in a real Duckietown environment]()
Physical Duckietown Environment
![Sim-to-Real evaluation environment for autonomous vehicles using the Gym-Duckietown simulator]()
Simulated Duckietown Environment
![Sim-to-Real semantic segmentation input representation for autonomous vehicles showing pixel-wise class labels]()
Semantic Segmentation Mask Example
![Sim-to-Real semantic segmentation network architecture for autonomous vehicles using FastSCNN]()
FastSCNN Network Architecture
![Sim-to-Real segmentation mask generated in CARLA for autonomous vehicles using OpenDRIVE metadata]()
CARLA Semantic Segmentation Output
![Sim-to-Real semantic segmentation pipeline for autonomous vehicles showing RGB input and segmented output]()
FastSCNN Input–Output Illustration
![Reward Function Variables]()
Reward Function Variables
![Sim-to-Real convolutional feature extractor for autonomous vehicles using NatureCNN architecture]()
NatureCNN Feature Extractor
![Sim-to-Real evaluation environments for autonomous vehicles across CARLA, Gym-Duckietown, and physical Duckiebot]()
Multi-Stage Evaluation Environments
![CKA Feature Similarity Heatmaps]()
CKA Feature Similarity Heatmaps
## The challenges and the approach
A core challenge addressed in this work is the inherent difficulty in ensuring that solutions developed in simulation remain effective when deployed on physical hardware.
Discrepancies between simulated environments and real-world conditions can cause significant performance degradation, particularly for perception and control algorithms.
These challenges include visual domain shifts, differences in dynamics and sensor noise, and the need for robust generalization across environments. The study explored how intermediate simulation platforms can help anticipate real-world performance and provides insight into limitations of current sim-to-real transfer approaches, reinforcing the importance of iterative testing across simulation and physical testbeds.
The approach followed consisted of deploying identical software stacks across low-fidelity and high-fidelity simulators, followed by execution on physical Duckietown vehicles. Metrics are collected for perception accuracy, trajectory tracking error, control stability, and failure rates. Domain discrepancies are analyzed by isolating sensing noise, actuator modeling, latency, and environmental dynamics. Challenges include simulator parameter mismatch, sensor noise modeling, real-time constraints, and non-linear vehicle dynamics that are not fully captured in simulation.
## Conclusions
The findings suggest that it is feasible to train models in a high-fidelity simulator such as CARLA and use a low-fidelity simulator to estimate real- world performance, thereby providing an approximation of the Sim-to-Real gap. However, results obtained in the intermediate simulator are not sufficiently reliable to eliminate the need for real-world testing. Training in a low-fidelity simulator like Duckietown and evaluating in CARLA proved to be much less effective. This indicates that the proposed method is well-suited for high-to-low fidelity trans- fer like discussed above, but not the reverse. Future work should look at broadening this methodology by incorporating multiple training algorithms, simulators and environments.
## Project Report
##### Looking for similar projects?
Check out the following works on sim-to-real with Duckietown:
- [Reproducible and Low-cost Sim-to-Real Environment for Traffic Signal Control](https://duckietown.com/reproducible-sim-to-real-traffic-signal-control-environment/)
- [Simulation to Real Domain Adaptation for Lane Segmentation](https://duckietown.com/sim2real-lane-segmentation-via-domain-adaptation/)
- [Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Real ](https://duckietown.com/sim2real-transfer-of-multi-agent-policies-for-self-driving/)[](https://duckietown.com/deep-reinforcement-learning-for-autonomous-lane-following/)
## Sim-to-Real Transfer for Small Autonomous Vehicles: authors
[  ](https://www.linkedin.com/in/jurriaan-buitenweg/?locale=en_US)
[Jurriaan Buitenweg](https://www.linkedin.com/in/jurriaan-buitenweg/?locale=en_US "Jurriaan Buitenweg") is currently working as a Machine Learning Engineer at [Enjins](http://enjins.com/ "Enjins"), Netherlands.
[  ](https://www.linkedin.com/in/jurriaan-buitenweg/?locale=en_US)
Dr. [Cynthia C. S. Liem](https://www.linkedin.com/in/cynthialiem/?originalSubdomain=nl) is a tenured Associate Professor at the Multimedia Computing Group of [Delft University of Technology](https://www.linkedin.com/in/cynthialiem/?originalSubdomain=nl).
### Learn more
Duckietown is a modular, customizable platform for robotics and artificial intelligence education, enabling hands-on learning and real-world experimentation with autonomous systems.
Designed for teaching, learning, and research, Duckietown supports the full spectrum of autonomy development, from foundational computer science and robotics concepts to advanced AI and self-driving systems research.
These spotlight projects are shared to demonstrate how Duckietown bridges theory and practice in robotics and AI, empowering students to apply machine learning and autonomy techniques to physical robots while building practical skills valued in academic research and industry.
[ Get Started ](https://duckietown.com/guides/)
[ More Projects ](https://duckietown.com/best-robotics-and-ai-projects-for-learning-autonomy/)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Projects, Systems & Infrastructure
**Tags:** perception, project, sensor fusion, student project
---
### [DB21v3-J Duckiebot upgrade kit now available](https://duckietown.com/db21v3-j-duckiebot-upgrade-kit-is-out/)
**Published:** February 5, 2026
**Author:** Duckietown Admin
**Excerpt:** The DB21Jv3 Duckiebot upgrade kit improves the omnidirectional wheel, reduces assembly time and increases compatibility with a range of Jetson Nano kits.
**Content:**
# DB21v3-J Duckiebot upgrade kit now available
The DB21Jv3 Duckiebot upgrade kit improves the omnidirectional wheel, reduces assembly time and increases compatibility with a range of Jetson Nano kits.
The DB21v3-J Duckiebot upgrade kit increases Duckiebot lifespan, enhances compatibility with different Jetson Nano 4GB development kits, improves driving performance, and reduces chassis assembly time.
## Upgrading your Duckiebot from DB21-M or -J to DB21-Jv3
Building on the experience and feedback from users worldwide, Duckiebots have undergone [many design iterations](https://docs.duckietown.com/ente/duckietown-manual/99-further-reading/duckiebot-configurations-explained.html "Duckiebot models and configurations explained") throughout the years.
This chassis upgrade, from DB21M or -J to DB21(v3)-J, introduces long-awaited improvements to the Duckiebot DB21 design, leading to shorter assembly time, better driving performance, and an overall improved user experience.
##### New omnidirectional wheel
A new omnidirectional wheel replaces the previous metal one, providing the following advantages with respect to the (historical) previous model:
- **Improved rigidity**: Three points of contact with the chassis instead of two, for improved rigidity and overall better driving performance
- **Designed for maintenance**: the new omnidirectional wheel can be opened and cleaned, providing an opportunity for removing the gunk that naturally builds up inside the wheel. This increases the life span of the wheel, and to some extent of the whole robot
- **Uniform friction in all directions**: thanks to the symmetry design and the undeformable nature of the components, this wheel provides more isotropic performance with respect to the previous model, leading to less force disturbance on the chassis and overall better driving performances.
##### Easier assembly process
For those who have experienced building DB21M/J Duckiebots, the mechanical tolerances between the characteristic chassis design and the metal nuts occasionally led to frustration. By replacing the metal chassis assembly screws and bolts with Nylon ones, Duckiebots become:
- Faster to assemble: thanks to perfect fits
- More joyful to assemble: thanks to a more reproducible Duckiebot assembly experience
##### Compatible chassis
The upgraed chassis now supports multiple Jetson Nano variants, including Jetson Super Orin Nano, OKDOs C100 Jetson Nano 4GB development Kit, and Waveshare Jetson Dev Kit.
[ Preorder your Duckiebot upgrade Kit ](https://get.duckietown.com/products/duckiebot-upgrade-kit-db21m-or-db21j-to-db21j-v3)
### Learn more about Duckietown
[Duckietown](https://duckietown.com/) is a set of tools that enables hands-on robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
**Categories:** News
**Tags:** DB21Jv3, db21jv3 duckiebot upgrade kit, duckiebot, Duckiebot upgrade kit, improved chassis, omnidirectional wheel, Upgrade Kit
---
### [Integrated Benchmarking and Design for Reproducible and Accessible Evaluation of Robotic Agents](https://duckietown.com/integrated-benchmarking-and-design-for-reproducible-and-accessible-evaluation-of-robotic-agents/)
**Published:** November 4, 2020
**Author:** Duckietown Admin
**Excerpt:** Duckietown Autolabs enables reproducible evaluation or robotic agents with an accessible setup, providing means for an integrated benchmarking system within the Duckietown ecosystem.
**Content:**
- Integrated Benchmarking and Design for Reproducible and Accessible Evaluation of Robotic Agents
- Jacopo Tani, Andrea F. Daniele, Gianmarco Bernasconi, Amaury Camus, Aleksandar Petrov, Anthony Courchesne, Bhairav Mehta, Rohit Suri, Tomasz Zaluska, Matthew R. Walter, Emilio Frazzoli, Liam Paull, Andrea Censi
- [ 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) October 25-29, 2020, Las Vegas, NV, USA (Virtual) ](https://arxiv.org/abs/2009.04362)
- [ ArXiv version download: arXiv:2009.04362v1 ](https://arxiv.org/pdf/2009.04362)
- [ Find the code here ](https://github.com/duckietown)
## Integrated Benchmarking and Design for Reproducible and Accessible Evaluation of Robotic Agents
### Why is this important?
As robotics matures and increases in complexity, it is more necessary than ever that robot autonomy research be **reproducible**.
Compared to other sciences, there are specific challenges to benchmarking autonomy, such as the complexity of the software stacks, the variability of the hardware and the reliance on data-driven techniques, amongst others.
We describe a new concept for reproducible robotics research that integrates development and benchmarking, so that reproducibility is obtained **by design** from the beginning of the research/development processes.
We first provide the overall conceptual objectives to achieve this goal and then a concrete instance that we have built: *the DUCKIENet*.
### The Duckietown Automated Laboratories (Autolabs)
One of the central components of this setup is the **Duckietown Autolab** (DTA), a remotely accessible standardized setup that is itself also relatively low-cost and reproducible.
DTAs include an off-the-shelf camera-based localization system. The accessibility of the hardware testing environment through enables experimental benchmarking that can be performed on a network of DTAs in different geographical locations.
### The DUCKIENet
When evaluating agents, careful definition of interfaces allows users to choose among local versus remote evaluation using simulation, logs, or remote automated hardware setups. The Decentralized Urban Collaborative Benchmarking Environment Network (DUCKIENet) is an instantiation of this design based on the Duckietown platform that provides an **accessible** and reproducible framework focused on autonomous vehicle fleets operating in model urban environments.
The DUCKIENet enables users to develop and test a wide **variety** of different algorithms using available resources (simulator, logs, cloud evaluations, etc.), and then deploy their algorithms locally in simulation, locally on a robot, in a cloud-based simulation, or on a real robot in a remote lab. In each case, the submitter receives feedback and scores based on well-defined metrics.
### Validation
We validate the system by analyzing the repeatability of experiments conducted using the infrastructure and show that there is low variance across different robot hardware and across different remote labs. We built DTAs at the Swiss Federal Institute of Technology in Zurich (ETHZ) and at the Toyota Technological Institute at Chicago (TTIC).
### Conclusions
Our contention is that there is a need for stronger efforts towards reproducible research for robotics, and that to achieve this we need to consider the evaluation in equal terms as the algorithms themselves. In this fashion, we can obtain reproducibility by design through the research and development processes. Achieving this on a large-scale will contribute to a more systemic evaluation of robotics research and, in turn, increase the progress of development.
If you found this interesting, you might want to:
[ Join the AI-DO ](https://www.duckietown.com/research/ai-driving-olympics)
[ Get your Autolab ](https://get.duckietown.com/)
[ Join our Slack ](https://join.slack.com/t/duckietown/shared_invite/zt-72tpbfth-xsfmv5iDAodqJ6eTFhjd4A)
**Categories:** paper, Research
**Tags:** autolab, Duckienet, IROS2020, paper, research
---
### [Safe Reinforcement Learning (RL) Thesis Project](https://duckietown.com/safe-reinforcement-learning-rl-duckietown-thesis-project/)
**Published:** May 17, 2024
**Author:** Duckietown Admin
**Excerpt:** "Safe Reinforcement Learning (Safe-RL)" explores using Deep Q Learning to train Duckiebots to perform lane following. Reproduce these results with Duckietown.
**Content:**
# Safe Reinforcement Learning (Safe-RL) in Duckietown
##### Project Resources
- **Objective**: Implement safe reinforcement learning (Safe-RL) to train Duckiebots at follow a lane, while keeping the robots within the boundaries of the road.
- **Approach**: Deep Q Learning
- **Authors**: Jan Steinmüller, Dr. Amr Alanwar Abdelhafez
[ Presentation ](https://github.com/Janst1000/Safe-RL-Duckietown/blob/v2/docs/Safe-RL-Duckietown.pdf)
[ Report ](https://github.com/Janst1000/Safe-RL-Duckietown/blob/v2/docs/Safe_Reinforcement_Learning_in_Duckietown.pdf)
[ Code ](https://github.com/Janst1000/Safe-RL-Duckietown)
[ Authors ](#authors)
## Safe-Reinforcement Learning (Safe-RL): Project Description
**Safe-RL Duckietown Project** – In his thesis titled “Safe-RL-Duckietown“, Jan Steinmüller used safe reinforcement learning to train Duckiebots to follow a lane while keeping said robots safe during training.
Safe Reinforcement Learning involves learning policies that maximize expected returns while ensuring reasonable system performance and adhering to safety constraints throughout both the learning and deployment phases. Reinforcement learning is a machine learning paradigm where agents learn to make decisions by maximizing cumulative rewards through interaction with an environment, without the necessity for training data or models.
The final result was a trained agent capable of following lanes while avoiding unsafe positions.
This is an open source project, and can be reproduced and improved upon through the Duckietown platform.
## Safe Reinforcement Learning: Project Highlights
Here is a visual tour of the work of the author.
[Check out the documents](#links "project resources") for more details.
![Implementation of the safe reinforcement learning (Safe-RL) Duckietown project]()
Implementation of the Safe-RL Duckietown project
![safe reinforcement learning in Duckietown project: Safety Layer Description]()
Safety Layer Description
![Process diagram of action selection and safety layer in the safe reinforcement learning project using Duckietown]()
Process diagram of action selection and safety layer
![Results of the safe reinforcement learning (Safe-RL) Duckietown Project]()
Results of the Safe-RL Duckietown Project
## Safe Reinforcement Learning: Results and Conclusions
Based on the results, it can be concluded that there is no disadvantage to using a safety layer when doing reinforcement learning since execution time is very similar. Moreover, the dramatically improved safety of the vehicle is helpful for the robot’s training as fewer actions with lower or even negative rewards will be executed. Because of this, reinforcement learning agents with safety layers learn faster and reduce the number of unsafe actions that are being executed.
Unfortunately, manual observation and intervention by the user were still necessary, however, the frequency was clearly reduced which further improved learning as the robots in testing did not know if an outside intervention was done which could result in an action being rewarded incorrectly.
It was also concluded that this project did not reach perfect safety with the implementation. Therefore a fully autonomous reinforcement learning training without any human intervention has not yet been achieved. A lot of improvement factors have been found that can further improve the safety and recovery rate. Additionally, some major problems which are not direct results of the reinforcement learning or safety layer have been identified.
These problems could be attempted to be fixed in different ways like improving the open source implementations of lane filter nodes or adding more sensors or cameras to the robot in order to extend the input data to the agent. Another area that was untouched during the research of this project was other vehicles inside the current lane. The safety layer could potentially be extended to also include safety features that should keep the robot safe from hitting other vehicles.
Read the full report [here](https://github.com/Janst1000/Safe-RL-Duckietown/blob/v2/docs/Safe_Reinforcement_Learning_in_Duckietown.pdf).
## Project Author
[  ](https://www.linkedin.com/in/suri-rohit/)
[Jan Steinmüller](https://www.linkedin.com/in/jan-steinm%C3%BCller-a0463a229 "Rohit Suri") is a computer science student working in the computer networks and information security research group at [Hochschule Bremen](https://www.hs-bremen.de/ "Venti Technologies") in Germany.
[  ](https://www.linkedin.com/in/suri-rohit/)
[Dr. Amr Alanwar](https://www.linkedin.com/in/amralanwar/ "Dr. Amr Alanwar") is an Assistant Professor at the [Technical University of Munich](https://www.tum.de/ "Technical University of Munich") (TUM).
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ People of Duckietown interviews ](https://duckietown.com/news/people-of-duckietown/)
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/research/papers)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Control & Reinforcement Learning, education, Projects
**Tags:** AI robotics, cSLAM, hands-on, localization, project, robot, robot autonomy, robotics, SLAM
---
### [Reinforcement Learning for the Control of Autonomous Robots](https://duckietown.com/reinforcement-learning-for-the-control-of-autonomous-robots/)
**Published:** November 6, 2024
**Author:** Duckietown Admin
**Excerpt:** This thesis applies Reinforcement Learning (RL) for autonomous lane-keeping and YOLO v5 obstacle detection in Duckietown, achieving safe navigation.
**Content:**
# Reinforcement Learning for the Control of Autonomous Robots
##### Project Resources
- **Objective**: Develop and evaluate reinforcement learning (RL) techniques for safe and autonomous navigation in any Duckietown
- **Approach**: Develop, train and test RL algorithms including Deep Q-Networks (DQN), Deep Deterministic Policy Gradient (DDPG), and Proximal Policy Optimization (PPO), for autonomous lane-keeping and obstacle detection on a DB21 Duckiebot.
- **Authors**: Bruno Fournier, Sébastien Biner
[ Report ](#thesis)
[ Final Result ](#project-result)
[ Authors ](#authors)
## RL on Duckiebots - Project highlights
Here is a visual tour of the authors’ work on implementing reinforcement learning in Duckietown.
![Diagram illustrating the basic principle of reinforcement learning applied to autonomous driving, showing an agent interacting with the environment, making decisions based on rewards and feedback.]()
Figure 1. Principle of Reinforcement Learning in Autonomous Driving.
![Diagram showing the application of reinforcement learning in the Duckietown environment, with a Duckiebot navigating simulated roadways based on RL feedback.]()
Figure 2. Reinforcement Learning in the Duckietown Environment.
![Comparison diagram showing the differences between Q-learning and Deep Q-Networks (DQN).]()
Figure 3. Q-learning vs. Deep Q-Networks (DQN).
![Diagram depicting the learning process with the Deep Q-Network (DQN) model, showing how actions are taken based on state inputs and updated using Q-value estimations.]()
Figure 4. Learning Process with the Deep Q-Network (DQN) Model.
![Diagram illustrating the architecture of the Deep Deterministic Policy Gradient (DDPG) algorithm, highlighting the actor and critic networks, experience replay, and target networks.]()
Figure 5. Architecture of the Deep Deterministic Policy Gradient (DDPG) Algorithm.
![Image of a simulation environment in Duckietown, displaying a virtual map with roads, intersections, and a Duckie.]()
Figure 6. Simulation Environment in Duckietown.
![Diagram of the Duckiebot test track with modular square elements, including straight lines, right-angle turns, and intersections, illuminated by two Walimex Pro LED lamps.]()
Figure 7. Modular Test Track for Duckiebot Driving Tests.
![Diagram illustrating the Duckiebot’s reward factors: the distance from the center of the lane (laned) and the angle relative to the lane’s centerline (laneθ), used in the DQN reward function.]()
Figure 8. Reward Factors for DQN in Duckiebot Navigation.
![Side-by-side images showing line detection in Duckietown before and after HSV parameter correction, illustrating improved clarity and accuracy of detected lines.]()
Figure 9. Line Detection Improvement with HSV Parameter Correction.
![Diagram illustrating the structure of the PA2 DQN model, showing the pre-processing of RGB images before they are fed into the neural network for reinforcement learning.]()
Figure 10. DQN Model Structure.
![Aerial view of the "loop_empty" training map used for DQN model training, featuring straight sections and both left and right turns.]()
Figure 11. Training Map for DQN.
![Illustration highlighting the differences between simulation and reality in the context of Duckietown, including variations in color tones, camera angles, and environmental objects.]()
Figure 12. Differences Between Simulation and Reality.
![Graph showing the average reward and average episode length for the DQN model in PA2 over multiple training episodes.]()
Figure 13. DQN (PA2): Average Reward and Average Episode Length.
![Graph showing episode-based rewards during the first phase of DDPG training.]()
Figure 14. DDPG Training 1: Episode-Based Rewards.
![Graph displaying episode-based rewards during the second phase of DDPG training.]()
Figure 15. DDPG Training 2: Episode-Based Rewards.
![Graph showing the average rewards achieved during the training process.]()
Figure 16. Average Rewards During Training.
![Graph showing the average distance traveled during each episode throughout the training.]()
Figure 17. Average Distance Traveled During Episodes.
![Graph showing the evolution of the agent's speed throughout the training process.]()
Figure 18. Evolution of the Agent's Speed.
![Graph showing the average reward and average episode length during Trial 1 of training.]()
Figure 19. Trial 1: Average Reward and Average Episode Length.
![Graph showing the average reward and average episode length during Trial 2 of training.]()
Figure 20. Trial 2: Average Reward and Average Episode Length.
![Graph showing the average reward and average episode length during Trial 3 of training.]()
Figure 21. Trial 3: Average Reward and Average Episode Length.
![Graph showing the average reward and average episode length during Trial 4 of training.]()
Figure 22. Trial 4: Average Reward and Average Episode Length.
![Visualization of the agent's trajectory on the evaluation track during testing.]()
Figure 23. Agent Trajectory on the Evaluation Track.
![Graph showing the average reward and average episode length during Trial 5 of training.]()
Figure 24. Trial 5: Average Reward and Average Episode Length.
![Graph showing the average reward and average episode length during Trial 6 of training.]()
Figure 25. Trial 6: Average Reward and Average Episode Length.
![Visualization of the robot's trajectory as it negotiates a bend on the track.]()
Figure 26. Trajectory Taken by the Robot to Negotiate a Bend.
![Graph showing the evolution of the safety factor throughout the training process.]()
Figure 27. Evolution of the Safety Factor.
## Why reinforcement learning for the control of Duckiebots in Duckietown?
This thesis explores the use of reinforcement learning (RL) techniques to enable autonomous navigation in the Duckietown. Reinforcement learning is a type of machine learning where an agent learns to make decisions by performing actions in an environment and receiving feedback through rewards or penalties. The goal is to maximize long-term rewards.
This work focuses on implementing and comparing various RL algorithms—specifically Deep Q-Network (DQN), Deep Deterministic Policy Gradient (DDPG), and Proximal Policy Optimization (PPO) – to analyze performance in autonomous navigation. RL enables agents to learn behaviors by interacting with their environment and adapting to dynamic conditions. The PPO model was found demonstrating smooth driving using grayscale images for enhanced computational efficiency.
Another feature of this project is the integration of YOLO v5, an object detection model, which allowed the Duckiebot to recognize and stop for obstacles, improving its safety capabilities. This integration of perception and RL enabled the Duckiebot not only to follow lanes but also to navigate autonomously, making ‘real-time’ adjustments based on its surroundings.
By transferring trained models from simulation to physical Duckiebots (Sim2Real), the thesis evaluates the feasibility of applying these models to real-world autonomous driving scenarios. This work showcases how reinforcement learning and object detection can be combined to advance the development of safe, autonomous navigation systems, providing insights that could eventually be adapted for full-scale vehicles.
[ Learn about Implementing Machine Learning with Duckietown ](https://duckietown.com/educational-resources/#machine-learning)
## Reinforcement learning for the control of Duckiebots in Duckietown - the challenges
Implementing reinforcement learning, in this project faced a number of challeneges summarized below –
- **Transfer from Simulation to Reality (Sim2Real):** Models trained in simulations often encountered difficulties when applied to real-world Duckiebots, requiring adjustments for accurate and stable performance.
- **Computational Constraints:** Limited processing power on the Duckiebots made it challenging to run complex RL models and object detection algorithms simultaneously.
- **Stability and Safety of Learning Models:** Guaranteeing that the Duckiebot’s actions were safe and did not lead to erratic behaviors or collisions required fine-tuning and extensive testing of the RL algorithms.
- **Obstacle Detection and Avoidance:** Integrating YOLO v5 for obstacle detection posed challenges in ensuring smooth integration with RL, as both systems needed to work harmoniously for obstacle avoidance.
These challenges were addressed through algorithm optimization, iterative model testing, and adjustments to the hyperparameters.
## Reinforcement learning for the control of Duckiebots in Duckietown: Results
## Reinforcement learning for the control of Duckiebots in Duckietown: Authors
[  ](https://www.linkedin.com/in/bruno-fournier-3305802b5/)
[Bruno Fournier](https://www.linkedin.com/in/bruno-fournier-3305802b5/) is currently pursuing Master of Science in Engineering, Data Science at the [HES-SO Haute école spécialisée de Suisse occidentale](http://www.hes-so.ch/), Switzerland.
[  ](https://www.linkedin.com/in/s%C3%A9bastien-biner-802376259/)
[Sébastien Biner](https://www.linkedin.com/in/s%C3%A9bastien-biner-802376259/) is currently pursuing Bachelor of Science in Automotive and Vehicle Technology at the [Berner Fachhochschule BFH](https://www.bfh.ch/de/), Switzerland.
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Control & Reinforcement Learning, Projects
**Tags:** project, student project
---
### [Deep Reinforcement Learning for Autonomous Lane Following](https://duckietown.com/deep-reinforcement-learning-for-autonomous-lane-following/)
**Published:** February 2, 2025
**Author:** Duckietown Admin
**Excerpt:** This project uses deep reinforcement learning for autonomous lane following, tackling sim-to-real challenges with domain adaptation and vision-based control.
**Content:**
# Deep Reinforcement Learning for Autonomous Lane Following
##### Project Resources
- **Objective**: Develop a deep reinforcement learning model for autonomous lane following with sim-to-real transfer.
- **Approach**: Train a reinforcement learning agent in simulation using an autoencoder-based feature extraction pipeline and deploy it on a real Duckiebot with domain adaptation techniques.
- **Authors**: Mickyas Tamiru Asfaw, David Bertoin, Valentin Guillet
[ Report ](#thesis)
[ University ](https://www.grenoble-inp.fr/)
[ Authors ](#authors)
## Project highlights
Here is a visual tour of the author’s work on implementing deep reinforcement learning for autonomous lane following in Duckietown.
![Deep Reinforcement Learning for Autonomous Lane Following]()
Figure 1. Deep Reinforcement Learning Architecture for Lane Following.
![Basic autoencoder architecture used in deep reinforcement learning for feature extraction and dimensionality reduction.]()
Figure 2. Autoencoder Architecture for Deep Reinforcement Learning.
![Image preprocessing and reconstruction using an autoencoder for deep reinforcement learning in autonomous lane following.]()
Figure 3. Image Preprocessing and Autoencoder Reconstruction.
## Deep reinforcement learning for autonomous lane following in Duckietown: objective and importance
Would it not be great if we could train an end-to-end neural network in simulation, plug it in the physical robot and have it drive safely on the road?
Inspired by this idea, Mickyas worked to implement deep reinforcement learning (DRL) for autonomous lane following in Duckietown, training the agent using sim-to-real transfer.
The project focuses on training DRL agents, including Deep Deterministic Policy Gradient ([DDPG](https://www.mathworks.com/help/reinforcement-learning/ug/ddpg-agents.html)), Twin Delayed DDPG ([TD3](https://spinningup.openai.com/en/latest/algorithms/td3.html)), and Soft Actor-Critic ([SAC](https://spinningup.openai.com/en/latest/algorithms/sac.html)), to learn steering control using high-dimensional camera inputs. It integrates an autoencoder to compress image observations into a latent space, improving computational efficiency.
The hope is for the trained DRL model to generalize from simulation to real-world deployment on a Duckiebot. This involves addressing domain adaptation, camera input variations, and real-time inference constraints, amongst other implementation challenges.
Autonomous lane following is a fundamental component of self-driving systems, requiring continuous adaptation to environmental changes, especially whn using vision as main sensing modality. This project identifies limitations in existing DRL algorithms when applied to real-world robotics, and explores modifications in reward functions, policy updates, and feature extraction methods analyzing the results through real world experimentation.
[ Learn Machine Learning with Duckietown ](https://duckietown.com/educational-resources/#machine-learning)
## The method and challenges in implementing deep reinforcement learning in Duckietown
The method involves training a DRL agent in a simulated Duckietown environment ([Gym Duckietown Simulator](https://github.com/duckietown/gym-duckietown "Gym Duckietown Simulator")) using an autoencoder for feature extraction.
The encoder compresses image data into a latent space, reducing input dimensions for policy learning. The agent receives sequential encoded frames as observations and optimizes steering actions based on reward-driven updates. The trained model is then transferred to a real Duckiebot using a ROS-based communication framework.
Challenges for pulling this off include accounting for discrepancies between simulated and real-world camera inputs, which affect performance and generalization. Differences in lighting, surface textures, and image normalization require domain adaptation techniques.
Moreover, computational limitations on the Duckiebot prevent direct onboard execution, requiring a distributed processing setup.
Reward shaping influences learning stability, and improper design of the reward function leads to policy exploitation or suboptimal behavior. Debugging DRL models is complex due to interdependencies between network architecture, exploration strategies, and training dynamics.
The project addresses these challenges by refining preprocessing, incorporating domain randomization, and modifying policy structures.
## Deep reinforcement learning for autonomous lane following: full report
## Deep reinforcement learning for autonomous lane following in Duckietown: Authors
[  ](https://www.linkedin.com/in/mickyas-tamiru-asfaw-1409271a6/)
[Mickyas Tamiru Asfaw](https://www.linkedin.com/in/mickyas-tamiru-asfaw-1409271a6/) is currently working as an AI Robotics and Innovation Engineer at the [CESI lineact laboratory](https://lineact.cesi.fr/en/), France.
[  ](https://www.linkedin.com/in/david-bertoin-876b485a/?originalSubdomain=fr)
[David Bertoin](https://www.linkedin.com/in/david-bertoin-876b485a/?originalSubdomain=fr) is currently working as a ML Applied Scientist at [Photoroom](https://photoroom.com/), France.
[  ](https://www.linkedin.com/in/valentinguillet/)
[Valentin Guillet](https://www.linkedin.com/in/valentinguillet/) is currently working as a Research engineer at [IRT Saint Exupéry](http://www.irt-saintexupery.com/), France.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Control & Reinforcement Learning, Projects
**Tags:** project, student project
---
### [Pure Pursuit Lane Following with Obstacle Avoidance](https://duckietown.com/pure-pursuit-lane-following-with-obstacle-avoidance/)
**Published:** June 16, 2025
**Author:** Duckietown Admin
**Excerpt:** This project implements adaptive pure pursuit control and deep learning-based obstacle detection for lane following and obstacle avoidance for Duckiebots.
**Content:**
# Pure Pursuit Lane Following with Obstacle Avoidance
##### Project Resources
- **Objective**: Develop a lane following and obstacle avoidance system using the pure pursuit control for Duckiebots. (DB18)
- **Approach**: Use an adaptive pure pursuit controller with dynamic speed tuning, integrated with vision-based deep-learning based object detection for obstacle-aware navigation.
- **Authors**: Soroush Saryazdi, Dhaivat Bhatt, Robotics and Embodied AI Lab (REAL), Université de Montréal and is also affiliated with MILA.
[ University ](https://www.umontreal.ca/)
[ Authors ](#authors)
[ Code ](https://github.com/saryazdi/Duckietown-Object-Detection-LFV)
## Project highlights
 Pure Pursuit Controller with Dynamic Speed and Turn Handling
 Pure Pursuit with Image Processing-Based Obstacle Detection
 Duckiebots Avoiding Obstacles with Pure Pursuit Control
## Pure Pursuit Lane Following with Obstacle Avoidance - the objectives
**Pure pursuit** is a geometric **path tracking** algorithm used in autonomous vehicle control systems. It calculates the curvature of the road ahead by determining a target point on the trajectory and computing the required angular velocity to reach that point based on the vehicle’s kinematics.
Unlike proportional integral derivative (PID) control, which adjusts control outputs based on continuous error correction, pure pursuit uses a lookahead point to guide the vehicle along a trajectory, enabling stable convergence to the path without oscillations. This method avoids direct dependency on derivative or integral feedback, reducing complexity in environments with sparse or noisy error signals.
This project aims to implement a pure pursuit-based **lane following system** integrated with obstacle avoidance for autonomous Duckiebot navigation. The goal is to enable real-time tracking of lane centerlines while maintaining safety through detection and response to dynamic obstacles such as other Duckiebots or cones.
The pipeline includes a modified ground projection system, an adaptive pure pursuit controller for path tracking, and both image processing and **deep learning-based object detection** modules for obstacle recognition and avoidance.
![Faster RCNN model architecture for pure pursuit and obstacle avoidance in Duckietown object detection]()
Faster RCNN Architecture with Feature Pyramid Network
![Object detection results using Faster RCNN for pure pursuit and obstacle avoidance in Duckietown with bounding boxes for duckiebots and traffic cones]()
Faster RCNN Detection Output with Bounding Boxes
![Duckietown object detection for pure pursuit and obstacle avoidance showing detected duckiebots and cones using deep learning]()
Detection Results for Obstacle Avoidance in Duckietown
[ Learn robot path planning with Duckietown ](https://duckietown.com/educational-resources/#planning)
## The challenges and approach
The primary challenges in this project include robust target point estimation under variable lighting and environmental conditions, **real-time object detection** with limited computational resources, and smooth trajectory control in the presence of dynamic obstacles.
The approach involves modular integration of perception, planning, and control subsystems.
For perception, the system uses both classical **image processing** methods and a trained deep learning model for object detection, enabling redundancy and simulation compatibility.
For planning and control, the pure pursuit controller dynamically adjusts speed and steering based on the estimated target point and obstacle proximity. Target point estimation is achieved through ground projection, a transformation that maps image coordinates to real-world planar coordinates using a calibrated camera model. Real-time parameter tuning and feedback mechanisms are included to handle variations in frame rate and sensor noise.
Obstacle positions are also ground-projected and used to trigger stop conditions within a defined safety zone, ensuring **collision avoidance** through reactive control.
##### Looking for similar projects?
Check out the following works on path planning with Duckietown:
- [Path Planning for Multi-Robot Navigation in Duckietown](https://duckietown.com/path-planning-for-multi-robot-navigation-in-duckietown/ "Path Planning for Multi-Robot Navigation in Duckietown")
- [Adaptive Lane Following with Auto-Trim tuning](https://duckietown.com/adaptive-lane-following-with-automatic-trim-calibration/)
- [Deep Reinforcement Learning for Autonomous Lane Following ](https://duckietown.com/deep-reinforcement-learning-for-autonomous-lane-following/)
## Pure Pursuit Lane Following with Obstacle Avoidance: Authors
[  ](https://saryazdi.github.io/)
[Soroush Saryazdi](https://saryazdi.github.io/ "Soroush Saryazdi") is currently leading the Neural Networks team at [Matic](https://maticrobots.com/ "Matic"), supervised by [Navneet Dalal](https://sites.google.com/view/navneetdalal/home "Navneet Dalal").
[  ](https://dhaivat1729.github.io/)
[Dhaivat Bhatt](https://dhaivat1729.github.io/ "Dhaivat Bhatt") is currently working as a Machine learning research engineer at Samsung AI centre, Toronto.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://duckietown.com/guides/)
[ More Projects ](https://duckietown.com/best-robotics-and-ai-projects-for-learning-autonomy/)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Control & Reinforcement Learning, Projects
**Tags:** navigation, path planning, project, student project
---
### [Introducing Autonomous Parking in Duckietown Cities](https://duckietown.com/autonomous-parking-in-duckietown/)
**Published:** May 27, 2024
**Author:** Duckietown Admin
**Excerpt:** This student project implements an autonomous parking solution, inclusive of parking lot design and autonomous behavior, for Duckiebots in Duckietown.
**Content:**
# Introducing Autonomous Parking in Duckietown Cities
##### Project Resources
- **Objective**: The goal of this project was to implement an autonomous parking solution for the Duckiebots.
- **Approach** : Design a Duckietown compliant parking lot, use only vision to maneuver and LEDs to coordinate with other Duckiebots
- **Authors**: Trevor Phillips, Vincenzo Polizzi, Linus Lingg
[ Final Result ](#project-result)
[ Report ](https://github.com/duckietown-ethz/proj-parking/blob/master/FinalReport.pdf)
[ Code ](https://github.com/duckietown-ethz/proj-parking)
[ Authors ](#authors)
## Why Autonomous Parking?
Parking is notoriously a hard task to master for many humans. Hence, students of the Autonomous Mobility on Demand course at ETH Zurich wanted to determine to what degree this applied to autonomous parking with Duckiebots.
The goal of the Autonomous Parking project was to design, implement, and test a complete autonomous parking solution compliant with the Duckietown ecosystem.
Duckiebots should be able to enter and exit a parking area, identify viable parking lots, actually park and exit their parking spot safely, and avoid collision with other Duckiebots during the entire process.
The vision is to integrate autonomous charging solutions into the parking area, so Duckiebots can charge themselves when needed.
## Autonomous parking in Duckietown: the challenges
Leveraging the Duckietown lane following vision baseline provided a basic infrastructure to build upon.
Some technical challenges specific to this projects were:
**Backward Lane Following:** Duckiebots must drive backward to exit the parking lots but only have cameras on the front. It is required to adjust the Duckiebot’s control system for stable backward driving, by changing the pose estimation process and re-tuning the PID controller.
**Dynamic Color Adaptation:** the new parking lot design introduced additional appearance specifications to the Duckietown city setup, such as blue lines identifying parking areas. Modifying the Duckiebots’ native lane detector to recognize blue lines in addition to yellow, red, and white, allows for additional flexibility in lane following based on specified colors.
**Time Slot Coordination**: Managing the availability of parking spaces is crucial to minimize the probability of collisions between Duckiebots. This project tackled this challenge by implementing a time-slot system to manage parking exits to prevent collisions, using red LEDs for signaling to other Duckiebots.
## Project Highlights
Here is a visual tour of the work of the authors.
[Check out the documents](#links "project resources") for more details!
![Autonomous parking lot with Duckiebots in Duckietown]()
Resulting parking lot area configuration within Duckietown. The parking lot is compliant with Duckietown appearance specifications, and features one entry/exit intersection. The parking lot closed-loop and modular configuration With the goal of enabling Duckiebots to autonomously enter, park, and exit parking spaces while avoiding collisions. This design exemplifies the project's focus on creating a functional and adaptable parking solution.
![Figure 2 illustrates a conceptual design of a parking area within the Duckietown environment. Comprising various standardized tiles, this depiction showcases a modular layout suitable for accommodating Duckiebots. The design includes six straight tiles, four curved left tiles, one three-way center tile, and two empty tiles. These components are arranged to form a configurable parking space that can be adapted to fit different spatial constraints. Notably, the configuration incorporates a T-intersection positioned such that Duckiebots are required to execute specific maneuvers upon entry and exit, ensuring adherence to the predetermined functionality of the parking area.]()
Design of a parking area within the Duckietown environment. Comprising various standardized tiles, this example showcases a modular layout suitable for accommodating Duckiebots. The design includes six straight tiles, four curved left tiles, one three-way center tile, and two empty tiles. Notably, the configuration incorporates a T-intersection positioned such that Duckiebots are required to execute specific maneuvers upon entry and exit, ensuring adherence to the predetermined functionality of the parking area.
![Autonomous parking in Duckietown, parking lot specifications]()
Parking spaces are delimited by a blue line with specified geometry. This configuration is functional to allow the Duckietown lane filter to determine the pose of the robot and facilitate precise entrance and exit from the parking space.
## Project Parking Results
(Turn on the sound for best experience!)
## Project Authors
[  ](https://www.linkedin.com/in/suri-rohit/)
[Trevor Phillips](https://www.linkedin.com/in/trevphil/) is a former Duckietown student, now a Machine Learning SWE at [Apple](https://www.apple.com/chde/) in Switzerland.
[Vincenzo Polizzi](https://www.linkedin.com/in/vincenzo-polizzi-602089146/) is a former Duckietown student, now a Ph. D. student at the [University of Toronto](https://www.utoronto.ca/ "University of Oxford"), Canada.
[Linus Lingg](https://www.linkedin.com/in/linus-lingg-203276154/?originalSubdomain=ch) is a former Duckietown student, now the Co-Founder and CTO of [bottleplus](https://bottleplus.com/) in Switzerland.
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ People of Duckietown interviews ](https://duckietown.com/news/people-of-duckietown/)
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Planning & Navigation, Projects
**Tags:** DB19, duckiebot, duckietown
---
### [Smart Lighting: Realistic Day and Night in Duckietown](https://duckietown.com/smart-lighting-autonomous-driving-realistic-day-and-night-in-duckietown/)
**Published:** October 11, 2024
**Author:** Duckietown Admin
**Excerpt:** What if Duckietowns had smart lighting, so that car and street light fields would combine dynamically for optimal visual perception?
**Content:**
# Smart Lighting: Realistic Day and Night in Duckietown
##### Project Resources
- **Objective**: City Realism: Characterising Night and Day Autonomous Driving Performance in the Duckietown Autolab
- **Approach**: Developing an automated street "smart" lighting system and control models, including a PI controller.
- **Authors**: David Müller
[ Report ](https://drive.google.com/file/d/1-TNi-ELFctx83-3CKzwSTJw_KgxDoQK0/view)
[ Final Result ](#project-result)
[ Authors ](#authors)
## Project Highlights
Here is the output of the authors’ work on smart lighting autonomous driving.
![A diagram showing the flow of nodes in the image processing pipeline, from camera input to lane detection using color filtering and line detection to enable smart lighting autonomous driving in Duckietown.]()
Figure 1. Image Processing Pipeline for Duckiebot Lane Detection.
![PID controller]()
Figure 2. PID controller.
![A diagram illustrating the open loop control system of the Duckietown street lighting system, showing the interaction between light sources and the image processing pipeline.]()
Figure 3. Open Loop Control of Duckietown Street Lighting System.
![A pair of wooden streetlight prototypes designed by Aurel Neff, positioned in Duckietown to provide lighting for smart lighting autonomous driving experiments.]()
Figure 4. Aurel Neff's Wooden Light Stands for Duckietown.
![Open loop of lighting system]()
Fig.5 Open loop of lighting system
![A control loop diagram showing the Duckiebot as the sole sensor for managing street lighting in Duckietown, using detected segments to control lighting conditions.]()
Figure 6. Control Loop with Duckiebot as the Only Sensor.
![A feedback loop diagram showing how the Duckiebot controls its own smart lighting system using detected lane segments and environmental lighting conditions.]()
Figure 7. Feedback Control Loop of Duckiebot Lighting System.
![A control loop diagram showing the watchtower's camera acting as a sensor to manage street lighting in Duckietown, influencing the Duckiebot's lane detection.]()
Figure 8. Control Loop with Watchtower Camera as Sensor.
![A feedback loop diagram illustrating how the watchtower's camera acts as a sensor to control the street lighting system in Duckietown.]()
Figure 9. Feedback Loop with Watchtower Camera as Sensor.
![A control loop diagram showing how the RGB sensor in the watchtower is used to manage street lighting conditions in Duckietown.]()
Figure 10. Control Loop Using RGB Sensor as Sensor.
![A graphical representation showing the effect of updated color ranges on the color detection capabilities of the Duckiebot in Duckietown.]()
Figure 13. Impact of Modified Color Ranges on Detection.
![Investigated lighting conditions in RGB space. The colored dots illustrates the colors of the edges of the grid.]()
Figure 14. Investigated lighting conditions in RGB space. The colored dots illustrates the colorsof the edges of the grid
![A comparison of (a) lane locations considered valid and (b) segments detected by the Duckiebot in Duckietown.]()
Figure 15. Filtering of Detected Segments in Duckietown.
![An image depicting the experimental setup used to assess optimal lighting conditions for the Duckiebot on a straight street in Duckietown.]()
Figure 16. Experimental Setup for Evaluating Optimal Lighting Conditions on a Straight Street.
![An image depicting the experimental setup used to assess the Duckiebot lighting system, highlighting that the WT04 watchtower is not operational.]()
Figure 17. Experimental Setup for Evaluating the Duckiebot Lighting System with Non-Functional WT04.
![Measured values for d and phi while the Duckiebot is following the lane]()
Figure 18. Lane following performance of the Duckiebot at different and changing light condition of the ceiling light and the street lighting system controlling the light on the streets of Duckietown
## Why day and night autonomous driving in Duckietown?
Autonomous driving is already inherently hard. Driving at night makes it even more challenging! This is why smart lighting is an interesting application that intersects with autonomous driving: having city infrastructure, such as traffic lights and watchtowers, generate dynamically varying light – only where and when they’re needed – to make driving at night not only possible but safe. Here are some reasons for which this project is interesting:
**Realistic driving scenarios**: autonomous driving systems must handle varying lighting conditions. Day and night cycles are just the beginning: transitions like sunrise or sunset make the spectrum of experimental corner cases more complex, hence Duckietown a valuable testbed.
**Robust lane-following capabilities**: developing an adaptive lighting system in which the city infrastructure “collaborates” with Duckiebot to provide optimal driving scenarios reinforces driving performances and general robustness for lane following.
**Decentralized control for scalability**: a decentralized approach to managing lighting implies that the system can be scalable across Duckietowns of arbitrary dimensions, making it more adaptable and resilient.
**Autonomous lighting management**: a responsive street lighting system, working in tandem with the Duckiebot’s onboard sensors, improves energy efficiency and ensures safety by adjusting to local lighting needs automatically.
[ Learn about Advanced Autonomy Approaches with Duckietown ](https://duckietown.com/educational-resources/#advanced-autonomy-approaches)
## Smart Lighting: Realistic Day and Night in Duckietown - the challenges
Implementing smart lighting in Duckietown to improve autonomous driving during day and night cycles presents several challenges. Here are a few examples:
**Hardware modifications**: while Duckiebots are equipped with controllable LEDs, city infrastructure does not possess lighting capabilities out of the box. The first step is integrating light sources in the design of Duckietown’s city infrastructure.
**Variable lighting conditions**: Duckiebots, which in this project rely uniquely on vision in their autonomy pipeline, must adapt to changing lighting conditions such as full darkness, sunrise, sunset, and artificial lighting, which impacts camera vision and lane detection accuracy.
**Decentralized control**: managing street lighting in a decentralized way across Duckietown ensures that each area adapts to its local lighting needs, compensating for example for the presence of passing Duckiebots with their own lights on. Join control algorithms including both city infrastructure and vehicle lighting intensity add complexity to the system’s design and coordination.
**Scalability**: the street lighting system must be scalable across the entire city, requiring a design that can be expanded without significant complications.
**Safe and reliable operation**: the system needs to be safe, adapting to issues such as occasional watchtower lighting source failure, while ensuring consistent lane-following performance.
## Smart Lighting: Realistic Day and Night in Duckietown: Results
## Smart Lighting: Realistic Day and Night in Duckietown: Authors
[  ](https://ch.linkedin.com/in/david-benjamin-m%C3%BCller)
[David Müller](https://ch.linkedin.com/in/david-benjamin-m%C3%BCller) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works as a Research Engineer at [Disney Research](https://disneyresearch.com/), Switzerland.
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Planning & Navigation, Projects
**Tags:** project, student project
---
### [Path Planning for Multi-Robot Navigation in Duckietown](https://duckietown.com/path-planning-for-multi-robot-navigation-in-duckietown/)
**Published:** April 5, 2025
**Author:** Duckietown Admin
**Excerpt:** A path planning algorithm that enables optimized multi-robot navigation in Duckietown, using nodegraph mapping and movement minimization techniques.
**Content:**
# Path Planning for Multi-Robot Navigation in Duckietown
##### Project Resources
- **Objective**: Design a scalable and optimal path planning system for autonomous robot navigation in Duckietown.
- **Approach**: Implemented a nodegraph-based path planning algorithm that maps quarter-tile nodes and calculates efficient movement sequences for robots.
- **Authors**: Alexander Hatteland, Marc-Philippe Frey, Demetris Chrysostomou
[ Report ](#thesis)
[ University ](https://ethz.ch/en.html)
[ Authors ](#authors)
[ Code ](https://github.com/duckietown-ethz/proj-goto-n)
## Project highlights
![Path planning nodegraph for Goto-N algorithm]()
Figure 1. Nodegraph of the Goto-N Pipeline
![Path planning script overview in Duckietown]()
Figure 2. Overview of Python Files
![Path planning tile node segmentation]()
Figure 3. Map Divided into 14 Tile Nodes
![Path planning legal node movements in Autolab]()
Figure 4. Allowable Moves from Nodes in Autolab
![Path planning optimal decisions per node]()
Figure 5. Optimal Moves by Termination Position
![Path planning accuracy measurement on Autobot]()
Figure 6. Final Precision Difference on Autobot
![Path planning move count per random trial]()
Figure 7. Movements per Trial with Random Start and End
![Path planning analysis with two robots]()
Figure 8. Total Movements with Two Robots
![Path planning evaluation with three robots]()
Figure 9. Total Movements with Three Robots
![Path planning average movement vs team size]()
Figure 10. Avg. Total Movements vs. Bots Used
![Path planning effort per robot based on team]()
Figure 11. Movements per Bot vs. Bots Used
![Path planning comparison of actual vs expected]()
Figure 12. Desired vs. Actual Termination Positions
[](https://duckietown.com/wp-content/uploads/2025/04/Nodegraph-of-the-Goto-N-Pipeline-png.avif) Figure 1. Nodegraph of the Goto-N Pipeline [](https://duckietown.com/wp-content/uploads/2025/04/Overview-of-Python-Files-png.avif) Figure 2. Overview of Python Files [](https://duckietown.com/wp-content/uploads/2025/04/Map-Divided-into-14-Tile-Nodes.avif) Figure 3. Map Divided into 14 Tile Nodes
[](https://duckietown.com/wp-content/uploads/2025/04/Allowable-Moves-from-Nodes-in-Autolab.avif) Figure 4. Allowable Moves from Nodes in Autolab [](https://duckietown.com/wp-content/uploads/2025/04/Optimal-Moves-by-Termination-Position.avif) Figure 5. Optimal Moves by Termination Position [](https://duckietown.com/wp-content/uploads/2025/04/Final-Precision-Difference-on-Autobot.avif) Figure 6. Final Precision Difference on Autobot
[](https://duckietown.com/wp-content/uploads/2025/04/Movements-per-Trial-with-Random-Start-and-End-png.avif) Figure 7. Movements per Trial with Random Start and End [](https://duckietown.com/wp-content/uploads/2025/04/Total-Movements-with-Two-Robots-png.avif) Figure 8. Total Movements with Two Robots [](https://duckietown.com/wp-content/uploads/2025/04/Total-Movements-with-Three-Robots-png.avif) Figure 9. Total Movements with Three Robots
[](https://duckietown.com/wp-content/uploads/2025/04/Avg.-Total-Movements-vs.-Bots-Used-png.avif) Figure 10. Avg. Total Movements vs. Bots Used [](https://duckietown.com/wp-content/uploads/2025/04/Movements-per-Bot-vs.-Bots-Used-png.avif) Figure 11. Movements per Bot vs. Bots Used [](https://duckietown.com/wp-content/uploads/2025/04/Desired-vs.-Actual-Termination-Positions-png.avif) Figure 12. Desired vs. Actual Termination Positions
## Path planning for multi-robot navigation in Duckietown - the objectives
Navigating Duckietown should not feel like solving a maze blindfolded!
The “Goto-N” path planning algorithm gives Duckiebots the map, the plan, and the smarts to take the optimal path from here to there, without wandering around by turning the map into a graph and every turn into a calculated choice.
While Duckiebots have long been able to follow lanes and avoid obstacles, truly strategic navigation, thinking beyond the next tile, toward a distant goal, requires a higher level of reasoning. In a dynamic Duckietown, robots need more than instincts. They need a plan.
This project introduces a node-based path-planning system that represents Duckietown as a graph of interconnected positions. Using this abstraction, Duckiebots can evaluate both allowable and optimal routes, adapt to different goal positions, and plan their moves intelligently.
The Goto-N project integrates several key concepts like:
- **Nodegraph representation**: transforms the tile-based Duckietown map into a graph of quarter-tile nodes, capturing all possible robot positions and transitions.
- **Allowable and optimal move generation**: differentiates between all legal movements and the most efficient moves toward a goal, supporting informed decision-making.
- **Termination-aware planning**: computes optimal actions relative to a chosen destination, enabling precise goal-reaching behaviors.
- **Multi-robot scalability**: validates the planner across one, two, and three Duckiebots to assess coordination, efficiency, and performance under shared conditions.
- **Real-world implementation and validation**: demonstrates the effectiveness of Goto-N through trials in the Autolab, comparing planned movements to real robot behavior.
[ Learn robot path planning with Duckietown ](https://duckietown.com/educational-resources/#planning)
[ Check out Goto-1: Planning with Dijkstra ](https://duckietown.com/goto-1-planning-with-dijkstra/)
## The challenges and approach
Navigating Duckietown poses several technical challenges: translating a continuous environment into a discrete planning space, handling edge cases like partial tile positions, and enabling efficient coordination among multiple autonomous agents.
The Goto-N project addresses these by discretizing the Duckietown map into a graph of ¼-tile resolution nodes, capturing all possible robot poses and orientations.
Using this representation, the system classifies allowable moves based on physical constraints and tile connectivity, then computes optimal moves to minimize distance or steps to a termination node using heuristics and precomputed lookup tables.
A Python-based pipeline then ingests the map layout, builds the nodegraph, and generates movement policies, which are then validated through simulated and physical trials. The system scales to multiple Duckiebots by assigning independent paths while analyzing overlap and bottlenecks in shared spaces, ensuring robust, efficient multi-robot planning.
## Path planning (Goto-n) in Duckietown: full report
The design and implementation of this path planning algorithm is documented in the following report.
## Path planning (goto-n) in Duckietown: Authors
[  ](https://www.linkedin.com/in/ahatteland/?originalSubdomain=no)
[Alexander Hatteland](https://www.linkedin.com/in/ahatteland/?originalSubdomain=no "Alexander Hatteland") is currently working as a Consultant at [Boston Consulting Group](http://www.bcg.com/ "Boston Consulting Group") (BCG), Switzerland.
[  ](https://www.linkedin.com/in/marc-philippe-fre/?originalSubdomain=ch)
[Marc-Philippe Frey](https://www.linkedin.com/in/marc-philippe-fre/?originalSubdomain=ch "Marc-Philippe Frey") is currently working as a Consultant at [Boston Consulting Group](http://www.bcg.com/ "Boston Consulting Group") (BCG), Switzerland.
[  ](https://www.linkedin.com/in/demetris-chrysostomou/?originalSubdomain=cy)
[Demetris Chrysostomou](https://www.linkedin.com/in/demetris-chrysostomou/?originalSubdomain=cy "Demetris Chrysostomou") is currently a PhD candidate at [Delft University of Technology](http://www.tudelft.nl/ "Delft University of Technology"), Netherlands.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Planning & Navigation, Projects
**Tags:** navigation, path planning, project, student project
---
### [QuackCruiser: Autonomous Navigation with Dijkstra](https://duckietown.com/autonomous-navigation-in-duckietown-with-quackcruiser/)
**Published:** August 30, 2025
**Author:** Duckietown Admin
**Excerpt:** This project develops Duckietown Autonomous Navigation using Dijkstra planning, perception, and control for lane following, turning, and obstacle detection.
**Content:**
# QuackCruiser: Autonomous Navigation with Dijkstra
##### Project Resources
- **Objective**: Enable robust Duckietown autonomous navigation using integrated perception, planning, and control modules.
- **Approach**: Use AprilTag detection, YOLO-based obstacle recognition, Dijkstra planning, and ROS-based state machine coordination.
- **Authors**: Li Yunwen, Farian Keck, Jiranyi from ETH Zurich, Switzerland
[ University ](https://ethz.ch/en.html)
[ Authors ](#authors)
[ Code ](https://github.com/li-yunwen/DuckieTown)
## Autonomous navigation in Duckietown with QuackCruiser - objectives and approach
The objective of this project is to implement autonomous navigation in Duckietown by integrating perception, Dijkstra planning, and control into a [Duckiebot](https://get.duckietown.com/products/duckiebot-db21?_gl=1*iaobnm*_gcl_au*Mjg5Mzc4Mzg0LjE3NTc5NTA4NzcuOTgyNzQxMDAzLjE3NjI0MjM0NzAuMTc2MjQyMzQ3MA..*_ga*NzQyMDczNTE2LjE3NDE3OTE4NTg.*_ga_99KJ333WVW*czE3NjI0MjI5ODQkbzMzJGcxJHQxNzYyNDIzOTI3JGo1OCRsMCRoMTM2OTI2MjQ0NiRkSjJKRDRiSWFUcGdHUXotZkhOTUVMREZfTlJseVZPNE9odw.. "Duckiebot") (DB21J).
Localization at intersection is achieved using [AprilTag](https://staging-docs.duckietown.com/ente/opmanual-autolab/localization_apriltag_specs.html?_gl=1*iaobnm*_gcl_au*Mjg5Mzc4Mzg0LjE3NTc5NTA4NzcuOTgyNzQxMDAzLjE3NjI0MjM0NzAuMTc2MjQyMzQ3MA..*_ga*NzQyMDczNTE2LjE3NDE3OTE4NTg.*_ga_99KJ333WVW*czE3NjI0MjI5ODQkbzMzJGcxJHQxNzYyNDIzOTI3JGo1OCRsMCRoMTM2OTI2MjQ0NiRkSjJKRDRiSWFUcGdHUXotZkhOTUVMREZfTlJseVZPNE9odw.. "AprilTag") detection, [YOLO-ROS](https://github.com/leggedrobotics/darknet_ros "YOLO-ROS") is used for real-time obstacle recognition, and onboard sensors such as wheel encoders and IMU are used for odometry. These inputs provide both exteroceptive data (from the environment) and interoceptive data (from the robot itself), which are fused to estimate pose and environment state.
Planning is performed with [Dijkstra](https://en.wikipedia.org/wiki/Dijkstra "Dijkstra’s") planning algorithm, a graph search method that computes the shortest path on a grid-based map where intersections are nodes and lanes are edges with associated costs.
Control is implemented through [PID](https://en.wikipedia.org/wiki/Proportional%E2%80%93integral%E2%80%93derivative_controller "PID")-based lane following and parameterized turning services, where each maneuver is defined by velocity, radius, and execution time. A ROS state machine coordinates perception inputs and planning outputs to trigger the correct control actions in ‘real time’.
[ Learn about autonomous driving and April Tags detection in Duckietown ](https://duckietown.com/educational-resources/#education-materials-modcon)
## Autonomous navigation in Duckietown with QuackCruiser - highlights
## The challenges
The principal challenges in implementing this agent emerge from hardware calibration, computational limitations, and cross-module synchronization.
Wheel encoder calibration directly influences odometric drift, while camera calibration governs the reliability of AprilTag-based localization and lane geometry estimation. The deployment of CUDA-accelerated YOLO models within the ROS ecosystem introduces compatibility constraints across GPU drivers, compiler toolchains, and real-time inference pipelines, which collectively impose significant computational overhead on limited embedded resources.
At the system integration level, temporal synchronization across perception modules (AprilTag detection, obstacle detection) and control modules (lane following, turning) constitutes a critical factor, as phase offsets and latencies propagate into localization uncertainty and trajectory deviation.
Sensor fusion must accommodate inconsistency between odometry estimates and visual updates, with conflict resolution strategies directly shaping the stability of pose estimation.
Furthermore, the tuning of control gains and maneuver execution parameters remains non-trivial, since cumulative deviations over extended trajectories amplify minor discrepancies in actuation dynamics and timing precision.
![Duckietown Autonomous Navigation system architecture with perception, planning, control, and actuation modules]()
System Architecture for Duckietown Autonomous Navigation
![Duckietown Autonomous Navigation environment with AprilTag-based intersection and goal tag setup]()
Environment Setup for Duckietown Autonomous Navigation
![Duckietown Autonomous Navigation map graph for Dijkstra planning with start, goal, and path costs]()
Map Graph for Duckietown Autonomous Navigation
![Duckietown Autonomous Navigation sample terminal log showing planner, state machine, and lane following execution]()
Terminal Log for Duckietown Autonomous Navigation
##### Looking for similar projects?
Check out the following works on path planning with Duckietown:
- [Path Planning for Multi-Robot Navigation in Duckietown](https://duckietown.com/path-planning-for-multi-robot-navigation-in-duckietown/ "Path Planning for Multi-Robot Navigation in Duckietown")
- [Adaptive Lane Following with Auto-Trim tuning](https://duckietown.com/adaptive-lane-following-with-automatic-trim-calibration/)
- [Deep Reinforcement Learning for Autonomous Lane Following ](https://duckietown.com/deep-reinforcement-learning-for-autonomous-lane-following/)
## Autonomous navigation in Duckietown with QuackCruiser: authors
[  ](https://www.linkedin.com/in/yunwen-li-ba3a2b20b/)
[Yunwem Li](https://www.linkedin.com/in/yunwen-li-ba3a2b20b/ "Yunwem Li") is a Computer engineering graduate with a master’s in robotics from [ETH Zurich](https://ethz.ch/en.html "ETH Zurich"), Switzerland.
[  ](https://www.linkedin.com/in/fariankeck/?originalSubdomain=ch)
[Farian Keck](https://www.linkedin.com/in/fariankeck/?originalSubdomain=ch "Farian Keck") is currently working as an Autonomy intern at [Airbus Defence and Space](http://www.airbus.com/ "Airbus Defence and Space") , Switzerland.
[  ](https://github.com/jiranyi)
[Jiranyi](https://github.com/jiranyi "Jiranyi") has a master’s in robotics from [ETH Zurich](https://ethz.ch/en.html "ETH Zurich"), Switzerland.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://duckietown.com/guides/)
[ More Projects ](https://duckietown.com/best-robotics-and-ai-projects-for-learning-autonomy/)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Planning & Navigation, Projects
**Tags:** perception, project, sensor fusion, student project
---
### [Autonomous Navigation System Development in Duckietown](https://duckietown.com/autonomous-navigation-system-development-in-duckietown/)
**Published:** May 31, 2025
**Author:** Duckietown Admin
**Excerpt:** This project implements an Autonomous Navigation System using computer vision, and Dijkstra algorithm for precise lane following and safe intersection handling.
**Content:**
# Autonomous Navigation System Development in Duckietown
##### Project Resources
- **Objective**: Enable safe and reliable autonomous navigation for Duckiebots in Duckietowns.
- **Approach**: Use onboard sensors, computer vision, and Dijkstra algorithm to navigate lanes, intersections, and avoid obstacles.
- **Authors**: Julien-Alexandre Bertin Klein, Andrea Pellegrin, Fathia Ismail from the Technical University of Munich, Germany.
[ University ](https://www.tum.de/)
[ Authors ](#authors)
[ Code ](https://github.com/DuckietownTUM/QuackSquad)
[ Report ](#report)
## Project highlights
[](https://duckietown.com/wp-content/uploads/2025/05/Visual-representation-of-the-Duckiebots-operational-modes.avif) Visual representation of the Duckiebot’s operational modes [](https://duckietown.com/wp-content/uploads/2025/05/Following-the-road-using-computer-vision.avif) Following the road using computer vision
[](https://duckietown.com/wp-content/uploads/2025/05/Detecting-and-stopping-at-the-stop-line-png.avif) Detecting and stopping at the stop line [](https://duckietown.com/wp-content/uploads/2025/05/Detecting-intersections-and-executing-turns-png.avif) Detecting intersections and executing turns
[](https://duckietown.com/wp-content/uploads/2025/05/Transforming-Roads-into-a-Navigable-Network-png.avif) Transforming Roads into a Navigable Network [](https://duckietown.com/wp-content/uploads/2025/05/Navigation-using-Dijkstras-Algorithm.avif) Navigation using Dijkstra’s Algorithm
![Duckiebot operational modes in autonomous navigation system]()
Visual representation of the Duckiebot’s operational modes
![Duckietown testing track and web dashboard for autonomous navigation]()
Following the road using computer vision
![Camera and line detector for autonomous navigation]()
Detecting and stopping at the stop line
![AprilTag and line detector views for autonomous navigation]()
Detecting intersections and executing turns
![Dijkstra algorithm map for autonomous navigation]()
Transforming Roads into a Navigable Network
![Real-world intersection handling in autonomous navigation]()
Navigation using Dijkstra’s Algorithm
## Autonomous Navigation System Development in Duckietown - the objectives
The primary objective of this project is to develop and refine an Autonomous Navigation System within the Duckietown environment, leveraging ROS-based control and computer vision to enable reliable lane following and safe intersection navigation. This includes calibrating sensor inputs, particularly from the camera, IMU, and encoders, and integrating advanced algorithms such as Dijkstra algorithm for optimal path planning. The project aims to ensure that the Duckiebot can autonomously detect lanes, stop lines, and obstacles while dynamically computing the shortest path to any designated point within the mapped environment. Additionally, the system is designed to transition smoothly between operational states (lane following, intersection handling, and recovery) using a refined Finite State Machine approach, all while maintaining robust communication within the ROS ecosystem.
[ Learn robot path planning with Duckietown ](https://duckietown.com/educational-resources/#planning)
## Project Report
## The challenges and approach
The project faced several challenges, beginning with hardware constraints, such as the physical limitations of wheel traction and battery lifespan, which affected motion stability and operational time. The integration of various ROS packages, some with incomplete documentation and inconsistent coding practices, complicated the development of a reliable and maintainable codebase. The method adopted involved precise sensor calibration to ensure accurate perception and control, incorporating camera intrinsic and extrinsic calibration for improved visual data interpretation, and adjusting wheel parameters to maintain balanced motion. The lane following module required parameter tuning for gain, trim, and heading correction to adapt to Duckietown’s environment. The original FSM-based intersection navigation system was re-engineered due to unreliability in node transitions, replaced with a distance-based approach for intersection stops and turns, ensuring deterministic and reliable behavior. Dijkstra’s algorithm was implemented to create a structured graph representation of the city map, enabling dynamic path planning that adapts to real-time inputs from the perception system. Custom web dashboards built with React.js and roslibjs facilitated monitoring and debugging by providing live data feedback and control interfaces. Through this rigorous and iterative process, the project achieved a robust autonomous navigation system capable of precise path planning and safe maneuvering within Duckietown.
##### Did this work spark your curiosity?
Check out the follow works on path planning with Duckietown:
- [Path Planning for Multi-Robot Navigation in Duckietown](https://duckietown.com/path-planning-for-multi-robot-navigation-in-duckietown/ "Path Planning for Multi-Robot Navigation in Duckietown")
- [Monocular Navigation in Duckietown Using LEDNet Architecture](https://duckietown.com/monocular-navigation-in-duckietown-using-lednet-architecture/ "Monocular Navigation in Duckietown Using LEDNet Architecture")
- [Goto-1: Planning with Dijkstra](https://duckietown.com/goto-1-planning-with-dijkstra/ "Goto-1: Planning with Dijkstra")
## Autonomous Navigation System Development in Duckietown: Authors
[  ](https://www.linkedin.com/in/jabk/)
[Julien-Alexandre Bertin Klein](https://www.linkedin.com/in/jabk/ "Julien-Alexandre Bertin Klein") is currently a Bachelor of Science (BSc.), Information Engineering at the [Technical University of Munich](https://www.tum.de/ "Technical University of Munich"), Germany.
[  ](https://github.com/andreapellegrin)
[Andrea Pellegrin](https://github.com/andreapellegrin "Andrea Pellegrin") is currently a Bachelor of Science (BSc.), Information Engineering at the [Technical University of Munich](https://www.tum.de/ "Technical University of Munich"), Germany.
[  ](https://www.linkedin.com/in/fathia-ismail-466a9a28b)
[Fathia Ismail](https://www.linkedin.com/in/fathia-ismail-466a9a28b "Fathia Ismail") is currently a Bachelor of Science (BSc.), Information Engineering at the [Technical University of Munich](https://www.tum.de/ "Technical University of Munich"), Germany.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Planning & Navigation, Projects
**Tags:** navigation, path planning, project, student project
---
### [Features for Efficient Autonomous Navigation in Duckietown](https://duckietown.com/visual-feedback-for-autonomous-navigation-in-duckietown/)
**Published:** June 28, 2025
**Author:** Duckietown Admin
**Excerpt:** Students at TUM build on the Duckietown out-of-the-box autonomous navigation pipeline introducing features for a more complete driving experience.
**Content:**
# Features for Efficient Autonomous Navigation in Duckietown
##### Project Resources
- **Objective**: Improving the range of autonomous behaviors of physical Duckiebots in Duckietown.
- **Approach**: Improving and integrating novel control (pure pursuit), perception (YOLO), and navigation algorithms on the Duckietown out-of-the-box "lane following" visual pipeline for autonomy.
- **Authors**: Servesh Khandwe, Ayush Kumar, and Parth Karkar from the Technical University of Munich, Germany.
[ University ](https://www.tum.de/)
[ Authors ](#authors)
[ Code ](https://github.com/DuckietownTUM/SAPduckie)
[ Report ](#report)
## Project highlights
[](https://duckietown.com/wp-content/uploads/2025/06/SSD-Model-Performance-mAP.avif) SSD Model Performance (mAP) [](https://duckietown.com/wp-content/uploads/2025/06/SSD-Model-Loss-Over-Iterations-png.avif) SSD Model Loss Over Iterations
[](https://duckietown.com/wp-content/uploads/2025/06/Object-Detection-in-Duckietown.avif) Object Detection in Duckietown [](https://duckietown.com/wp-content/uploads/2025/06/Labels-of-the-baseline-Model-png.avif) Labels of the baseline Model
[](https://duckietown.com/wp-content/uploads/2025/06/Labels-of-the-new-fine-tuned-Model.avif) Labels of the new fine-tuned Model [](https://duckietown.com/wp-content/uploads/2025/06/Confusion-Matrix-of-the-baseline-Model-png.avif) Confusion Matrix of the baseline Model
[](https://duckietown.com/wp-content/uploads/2025/06/Confusion-Matrix-of-the-new-finetuned-Model-png.avif) Confusion Matrix of the new finetuned Model [](https://duckietown.com/wp-content/uploads/2025/06/Model-Performance-on-real-Duckietown-environment.avif) Model Performance on real Duckietown environment
[](https://duckietown.com/wp-content/uploads/2025/06/Annotations-format-used-by-YOLO-for-output.avif) Annotations format used by YOLO for output [](https://duckietown.com/wp-content/uploads/2025/06/Ground-Projection.avif) Ground Projection for Path Tracking
[](https://duckietown.com/wp-content/uploads/2025/06/Lane-Edge-detection-png.avif) Edge Detection of Lane Boundaries [](https://duckietown.com/wp-content/uploads/2025/06/Color-detector-for-stop-line-filter.avif) Color detector for stop line filter
[](https://duckietown.com/wp-content/uploads/2025/06/Bot-detector-for-Collision-avoidance.avif) Bot detector for Collision avoidance [](https://duckietown.com/wp-content/uploads/2025/06/April-Tag-sign-detection.avif) April Tag sign detection
## Visual Feedback for Autonomous Navigation in Duckietown - the objectives
This project from students at TUM (Technische Universität of Munich) builds on the preexisting Duckietown autonomy stack to add/reintegrate/improve upon much-needed autonomous navigation features: improved control (pure pursuit instead of PID), red stop line detection, AprilTag detection, intersection navigation, and obstacle detection (using YOLO v3), making Duckietowns more complex and interesting!
The resulting agent includes modules for lane following, stop line detection, and intersection handling using AprilTags, following the legacy infrastructure of Duckietown.
The autonomy pipeline relies heavily on vision as the primary means of perception: lane edges are projected from image space to the ground plane using inverse perspective mapping learned after running a camera calibration procedure.
The Duckiebot then estimates a dynamic target point by offsetting yellow or white lane markers depending on visibility. The curvature is computed based on the geometric relation between the Duckiebot and the goal point, and the steering command is derived from this curvature.
The Duckiebot velocity and angular velocity are then modulated using a second-degree polynomial function based on detected path geometry.
Visual input from an onboard monocular camera is processed through a lane filter with adaptive Gaussian variance scaling relative to frame timing.
When running by an intersection, stop lines are detected using [HSV color segmentation](https://medium.com/@Christian_Gunawan/image-segmentation-using-thresholding-hsv-color-space-segmentation-793722bb758d "HSV color segmentation"). AprilTag detection determines intersection decisions, with tag IDs mapped to turn directions.
Every module is implemented as an independent ROS package with dedicated launch files, coordinated via a central launch file. A [YOLOv3](https://docs.ultralytics.com/models/yolov3/ "YOLOv3") object detection model, trained on a custom Duckietown dataset, provides real-time obstacle recognition.
[ Learn robot path planning with Duckietown ](https://duckietown.com/educational-resources/#planning)
![SSD object detection performance for autonomous navigation using pure pursuit in Duckietown]()
SSD Model Performance (mAP)
![Loss graph of SSD model during autonomous navigation training using pure pursuit in Duckietown]()
SSD Model Loss Over Iterations
![SSD-based object detection output in autonomous navigation using pure pursuit in Duckietown]()
Object Detection in Duckietown
![Initial YOLO model label annotations in autonomous navigation using pure pursuit in Duckietown]()
Labels of the baseline Model
![Refined label annotations after YOLO model fine-tuning for autonomous navigation using pure pursuit in Duckietown]()
Labels of the new fine-tuned Model
![Confusion matrix showing classification results of the baseline YOLO model in autonomous navigation using pure pursuit in Duckietown]()
Confusion Matrix of the baseline Model
![Confusion matrix showing improved detection accuracy of the fine-tuned YOLO model in autonomous navigation using pure pursuit in Duckietown]()
Confusion Matrix of the new finetuned Model
![YOLO model output in real-time autonomous navigation using pure pursuit in Duckietown]()
Model Performance on real Duckietown environment
![Annotation format structure used by YOLO for object detection in autonomous navigation using pure pursuit in Duckietown]()
Annotations format used by YOLO for output
![Ground projection technique applied for lane tracking in autonomous navigation using pure pursuit in Duckietown]()
Ground Projection for Path Tracking
![Lane edge detection method supporting autonomous navigation using pure pursuit in Duckietown]()
Edge Detection of Lane Boundaries
![Color-based stop line detection system for autonomous navigation using pure pursuit in Duckietown]()
Color detector for stop line filter
![Real-time bot detection for collision avoidance in autonomous navigation using pure pursuit in Duckietown]()
Bot detector for Collision avoidance
![AprilTag detection system for intersection navigation in autonomous navigation using pure pursuit in Duckietown]()
April Tag sign detection
## The challenges and approach
One major hurdle was integrating object detection models like Single-Shot Detector (SSD) and YOLO with the Duckiebot’s ROS-based camera system.
While the SSD model was trained on a custom Duckietown dataset, ROS publisher-subscriber mismatches prevented live inference. Transitioning to the YOLO model involved adapting annotation formats and re-training for compatibility with the YOLO architecture. In lane following, the default controller from Duckietown demos showed high deviation, prompting the implementation of a modified pure pursuit approach.
Additional challenges arose from limited computational resources on the Duckiebot, with CPU overuse causing processing delays when running all modules concurrently. The approach focused on modular development, isolating lane following, stop line detection, and intersection navigation into separate ROS packages with fine-tuned parameters. The pure pursuit algorithm was adapted for ground-projected lane estimation, dynamic speed control, and target point calculation based on visible lane markers. Integration of AprilTag-based intersection logic and LED signaling provided directional control at intersections.
This structured, iterative methodology enabled real-time, vision-guided behavior while operating within the constraints.
## Project Report
##### Did this work spark your curiosity?
Check out the follow works on path planning with Duckietown:
- [Path Planning for Multi-Robot Navigation in Duckietown](https://duckietown.com/path-planning-for-multi-robot-navigation-in-duckietown/ "Path Planning for Multi-Robot Navigation in Duckietown")
- [Monocular Navigation in Duckietown Using LEDNet Architecture](https://duckietown.com/monocular-navigation-in-duckietown-using-lednet-architecture/ "Monocular Navigation in Duckietown Using LEDNet Architecture")
- [Goto-1: Planning with Dijkstra](https://duckietown.com/goto-1-planning-with-dijkstra/ "Goto-1: Planning with Dijkstra")
## Visual Feedback for Autonomous Navigation in Duckietown: Authors
[  ](https://www.linkedin.com/in/serveshkhandwe/?originalSubdomain=de)
[Servesh Khandwe](https://www.linkedin.com/in/serveshkhandwe/?originalSubdomain=de "Servesh Khandwe") is currently working as a Software Engineer at [Porsche Digital](https://www.porsche.digital/ "Porsche Digital"), Germany.
[  ](https://www.linkedin.com/in/ayushkumar13445/?originalSubdomain=de)
[Ayush Kumar](https://www.linkedin.com/in/ayushkumar13445/?originalSubdomain=de "Ayush Kumar") is currently working as a Research Assistant at [Fraunhofer IIS](http://www.iis.fraunhofer.de/en.html "Fraunhofer IIS"), Germany.
[  ](https://www.linkedin.com/in/fathia-ismail-466a9a28b)
[Parth Karkar](https://www.linkedin.com/in/parth-karkar-01724b1b9/?originalSubdomain=de "Parth Karkar") is currently working as an Analytical Consultant at [Mutares SE & Co. KGaA](http://www.mutares.de/ "Mutares SE & Co. KGaA"), Germany.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Planning & Navigation, Projects
**Tags:** navigation, path planning, project, student project
---
### [Monocular Navigation in Duckietown Using LEDNet Architecture](https://duckietown.com/monocular-navigation-in-duckietown-using-lednet-architecture/)
**Published:** November 30, 2024
**Author:** Duckietown Admin
**Excerpt:** This project is on monocular navigation in Duckietown using LEDNet, comparing its performance with vision transformer for lane-following and obstacle avoidance.
**Content:**
# Monocular Navigation in Duckietown Using LEDNet Architecture
##### Project Resources
- **Objective**: Autonomous lanel following and obstable avoidance in Duckietown using vision and machine learning.
- **Approach**: Use monocular vision and "LEDNet" with vision transformer models. Simulated tests evaluate LEDNet's high-resolution performance against vision transformer's low-resolution capabilities.
- **Authors**: Angelo R. Broere
[ Report ](#thesis)
[ University ](https://www.uva.nl/en)
[ Final Result ](#project-result)
[ Authors ](#authors)
## Project highlights
Here is a visual tour of the authors’ work on implementing monocular navigation using LEDNet architecture in Duckietown\*.
![ViT image segmentation outputs for Duckietown showing the effect of 1 block and 3 blocks in the model.]()
Figure 1. ViT Image Segmentation Outputs for Duckietown: Comparing 1 Block vs 3 Blocks.
![Illustration of an encoder-decoder architecture (SegNet) used for pixelwise segmentation for the monocular navigation project.]()
Figure 2. Encoder-Decoder Architecture (SegNet) for Pixelwise Segmentation.
![Visual representation of the LEDNet architecture showing its lightweight encoder-decoder structure.]()
Figure 3. The LEDNet Architecture.
![LEDNet image segmentation of Duckietown showing multi-scale feature pyramids for pixel-level attention.]()
Figure 4. LEDNet Image Segmentation of Duckietown.
![LEDNet loss graph showing the flattening of the loss curve after 200 epochs.]()
Figure 5. LEDNet Loss Graph.
![Simulated Duckietown map 'loop_empty' showing a simple layout with left and right bends.]()
Figure 6. Simulated Duckietown Map: 'loop\_empty'.
![Simulated Duckietown map 'loop_empty' with obstacles such as Duckiebots and rubber ducks.]()
Figure 7. Simulated Duckietown Map: 'loop\_empty' with Obstacles.
![Visual representation of the lane-following and obstacle-avoidance algorithm from Saavedra-Ruiz et al. (2022).]()
Figure 8. Lane-Following and Obstacle-Avoidance Algorithm (Saavedra-Ruiz et al., 2022).
![Comparison of image segmentations created by LEDNet, ViT 1 Block, and ViT 3 Blocks, highlighting the detection of small obstacles.]()
Figure 9. Image Segmentations: LEDNet vs. ViT 1 Block vs. ViT 3 Blocks.
\*Images from “Monocular Robot Navigation with Self-Supervised Pretrained Vision Transformers, M. Saavedra-Ruiz, S. Morin, L. Paull. ArXiv:
## Why monocular navigation?
Image sensors are ubiquitous for their well-known sensory traits (e.g., distance measurement, robustness, accessibility, variety of form factors, etc.). Achieving autonomy with monocular vision, i.e., using only one image sensor, is desirable, and much work has gone into approaches to achieve this task. Duckietown’s [first Duckiebot, the DB17](https://docs.duckietown.com/daffy/opmanual-duckiebot/preliminaries_hardware/duckiebot_configurations/index.html "Duckiebot configurations"), was designed with only a camera as sensor suite to highlight the importance of this challenge!
But images, due to the integrative nature of image sensors and the [physics of the image generation process](https://duckietown.com/educational-resources#education-materials-cv "Duckietown Computer Vision learning materials"), are subject to motion blur, occlusions, and sensitivity to environmental lighting conditions, which challenge the effectiveness of “traditional” computer vision algorithms to extract information.
In this work, the author uses “LEDNet” to mitigate some of the known limitations of image sensors for use in autonomy. LEDNet’s encoder-decoder architecture with high resolution enables lane-following and obstacle detection. The model processes images at high frame rates, allowing recognition of turns, bends, and obstacles, which are useful for timely decision-making. The resolution improves the ability to differentiate road markings from obstacles, and classification accuracy.
LEDNet’s obstacle-avoidance algorithm can classify and detect obstacles even at higher speeds. Unlike [Vision Transformers (wiki)](https://en.wikipedia.org/wiki/Vision_transformer "Vision Transformers, Wikipedia") (ViT) models, LEDNet avoids missing parts of obstacles, preventing robot collisions.
The model handles small obstacles by identifying them earlier and navigating around them. In the simulated Duckietown environment, LEDNet outperforms other models in lane-following and obstacle-detection tasks.
LEDNet uses “real-time” image segmentation to provide the Duckiebot with information for steering decisions. While the study was conducted in a simulation, the model’s performance indicates it would work in real-world scenarios with consistent lighting and predictable obstacles.
The next is to try it out!
[ Learn about Implementing Autonomous Navigation with Duckietown ](https://duckietown.com/educational-resources/#machine-learning)
## Monocular Navigation in Duckietown Using LEDNet Architecture - the challenges
In implementing monocular navigation in this project, the author faced several challenges:
1. **Computational demands**: LEDNet’s high-resolution processing requires computational resources, particularly when handling real-time image segmentation and obstacle detection at high frame rates.
2. **Limited handling of complex environments**: the lane-following and obstacle-avoidance algorithm used in this study does not handle crossroads or junctions, limiting the model’s ability to navigate complex road structures.
3. **Simulation vs. real-world application**: The study relies on a simulated environment where lighting, obstacle behavior, and road conditions are consistent. Implementing the system in the real world introduces variability in these factors, which affects the model’s performance.
4. **Small obstacle detection**: While LEDNet performs well in detecting small obstacles compared to ViT, the detection of small obstacles is still dependent on the resolution and segmentation quality.
## Project Report
## Project Author
[  ](https://www.linkedin.com/in/angelobroere/)
[Angelo Broere](https://www.linkedin.com/in/angelobroere/) is currently working as an Oproepkracht at Compressor Parts Service, Netherlands.
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Localization, Projects
**Tags:** project, student project
---
### [Extended Kalman Filter (EKF) SLAM for Duckiebots](https://duckietown.com/extended-kalman-filter-slam-for-duckiebots/)
**Published:** April 26, 2025
**Author:** Duckietown Admin
**Excerpt:** Implementing Extended Kalman Filter (EKF) SLAM on Duckiebots to enhance localization accuracy and map static landmarks using AprilTags and odometry.
**Content:**
# Extended Kalman Filter (EKF) SLAM for Duckiebots
##### Project Resources
- **Objective**: Implement an Extended Kalman Filter (EKF) SLAM system to estimate accurate Duckiebot pose and landmark positions in Duckietown environments.
- **Approach**: Fuse odometry and AprilTag landmark detections using EKF to achieve robust localization and mapping for small-scale autonomous Duckiebots.
- **Authors**: Amir Hossein Zamani, Léonard Oest O'Leary, Kevin Lessard from the University of Montreal.
[ University ](https://www.umontreal.ca/en/)
[ Authors ](#authors)
[ Code ](https://github.com/AHHHZ975/SLAM-Duckietown)
## Project highlights
> In SLAM, everything that can drift will drift, and the role of the filter is to drift more slowly than entropy.
![Extended Kalman Filter (EKF) SLAM system architecture overview]()
Figure 1. EKF-SLAM Architecture Overview
![Extended Kalman Filter (EKF) SLAM input and output fusion diagram]()
Figure 2. Extended Kalman Filter (EKF) SLAM input and output flow diagram
![Extended Kalman Filter (EKF) SLAM incorporating landmarks into odometry problem]()
Figure 3. Incorporating landmarks into the odometry problem
![Extended Kalman Filter (EKF) SLAM odometry trajectory after incorporating landmarks]()
Figure 4. Odometry after incorporating landmarks
![Extended Kalman Filter (EKF) SLAM odometry before incorporating landmarks]()
Figure 5. Odometry before incorporating landmarks
![Extended Kalman Filter (EKF) SLAM improved versus simple interpolation for trajectory estimation]()
Figure 6. EKF-SLAM Trajectory Interpolation
![The equations describe how the Duckiebot's position and orientation (x,y,θ) evolve over time based on its linear velocity (v) and angular velocity (ω).]()
Figure 7. EKF's prediction-correction workflow
![Extended Kalman Filter (EKF) SLAM AprilTag detection and pose estimation steps]()
Figure 8. EKF-SLAM AprilTags Detection
![The diagrams illustrates the coordinate transformations and the pose estimation workflow.]()
Figure 9. EKF-SLAM AprilTags Pose Estimation
![The visual illustrates the EKF's prediction-correction workflow, showing how odometry data predicts the next state and April tag measurements refine it. By iteratively fusing these data sources, the EKF achieves accurate localization and mapping, even under noisy or incomplete sensor inputs.]()
Figure 10. EKF-SLAM Motion Model Equations
![EKF-SLAM full version (motion+measurement model)]()
Figure 11. EKF-SLAM full version (motion+measurement model)
![EKF-SLAM measurement model (without motion model)]()
Figure 12. EKF-SLAM measurement model (without motion model)
![EKF-SLAM motion model (without measurement model)]()
Figure 13. EKF-SLAM motion model (without measurement model)
![Figure 14. Pose estimation using a circular interpolation]()
Figure 14. Pose estimation using a circular interpolation
![Pose estimation using a circular interpolation_DELTA_TIME=2]()
Figure 15. Pose estimation using a circular interpolation\_DELTA\_TIME=2
![Pose estimation using a linear interpolation]()
Figure 16. Pose estimation using a linear interpolation
![Figure 17. Pose estimation using a linear interpolation_DELTA_TIME=2]()
Figure 17. Pose estimation using a linear interpolation\_DELTA\_TIME=2
[](https://duckietown.com/wp-content/uploads/2025/04/EKF-SLAM-Architecture-Overview-png.avif) Figure 1. EKF-SLAM Architecture Overview [](https://duckietown.com/wp-content/uploads/2025/04/EKF-SLAM-Odometry-and-Landmarks-png.avif) Figure 2. Extended Kalman Filter (EKF) SLAM input and output flow diagram [](https://duckietown.com/wp-content/uploads/2025/04/Incorporating-landmarks-into-the-odometry-problem.avif) Figure 3. Incorporating landmarks into the odometry problem
[](https://duckietown.com/wp-content/uploads/2025/04/Odometry-after-incorporating-landmarks.avif) Figure 4. Odometry after incorporating landmarks [](https://duckietown.com/wp-content/uploads/2025/04/Odometry-before-incorporating-landmarks-png.avif) Figure 5. Odometry before incorporating landmarks [](https://duckietown.com/wp-content/uploads/2025/04/EKF-SLAM-Trajectory-Interpolation-png.avif) Figure 6. EKF-SLAM Trajectory Interpolation
[](https://duckietown.com/wp-content/uploads/2025/04/EKFs-prediction-correction-workflow-png.avif) Figure 7. EKF’s prediction-correction workflow [](https://duckietown.com/wp-content/uploads/2025/04/EKF-SLAM-AprilTags-Detection.avif) Figure 8. EKF-SLAM AprilTags Detection [](https://duckietown.com/wp-content/uploads/2025/04/EKF-SLAM-AprilTags-Pose-Estimation-png.avif) Figure 9. EKF-SLAM AprilTags Pose Estimation
[](https://duckietown.com/wp-content/uploads/2025/04/EKF-SLAM-Motion-Model-Equations-png.avif) Figure 10. EKF-SLAM Motion Model Equations [](https://duckietown.com/wp-content/uploads/2025/04/EKF-SLAM-full-version-motionmeasurement-model-png.avif) Figure 11. EKF-SLAM full version (motion+measurement model) [](https://duckietown.com/wp-content/uploads/2025/04/EKF-SLAM-measurement-model-without-motion-model-png.avif) Figure 12. EKF-SLAM measurement model (without motion model)
[](https://duckietown.com/wp-content/uploads/2025/04/EKF-SLAM-motion-model-without-measurement-model-png.avif) Figure 13. EKF-SLAM motion model (without measurement model) [](https://duckietown.com/wp-content/uploads/2025/04/Pose-estimation-using-a-circular-interpolation-png.avif) Figure 14. Pose estimation using a circular interpolation [](https://duckietown.com/wp-content/uploads/2025/04/Pose-estimation-using-a-circular-interpolation_DELTA_TIME2-png.avif) Figure 15. Pose estimation using a circular interpolation\_DELTA\_TIME=2
[](https://duckietown.com/wp-content/uploads/2025/04/Pose-estimation-using-a-linear-interpolation-png.avif) Figure 16. Pose estimation using a linear interpolation [](https://duckietown.com/wp-content/uploads/2025/04/Pose-estimation-using-a-linear-interpolation_DELTA_TIME2-png.avif) Figure 17. Pose estimation using a linear interpolation\_DELTA\_TIME=2
## Extended Kalman Filter (EKF) SLAM for Duckiebots - the objectives
This SLAM-Duckietown project addresses a famous challenge in robotics: concurrently estimating the agent’s pose and mapping the environment under uncertainty.
This project implements an Extended Kalman Filter (EKF) SLAM algorithm on Duckiebots (DB21-J4), combining odometry from wheel encoders and landmark observations from April tags.
The objective is to maintain an evolving posterior over the Duckiebot’s pose (x,y,θ) and landmark positions by recursively integrating noisy control inputs and observations.
This upgrade shifts Duckiebots from open-loop dead reckoning units into closed-loop, state-estimating agents. For Duckietown, it reinforces its use as an experimental ground for real-world robotics challenges, including data association, observability, filter consistency, and multi-sensor fusion.
[ Learn robot path planning with Duckietown ](https://duckietown.com/educational-resources/#planning)
## The challenges and approach
The system applies the EKF-SLAM pipeline in two stages: **motion prediction and measurement correction**.
**Prediction** propagates the robot’s belief through a non-holonomic kinematic model under process noise, using arc-based interpolation to reduce discretization error.
**Correction** incorporates April tag detections via a [Perspective-n-Point (PnP)](https://en.wikipedia.org/wiki/Perspective-n-Point "Perspective-n-Point (PnP)") solution, updating the state with landmark-relative observations under observation noise. The state vector grows dynamically as new landmarks are observed, and the covariance matrix tracks both robot and landmark uncertainty.
The technical **challenges** include maintaining filter consistency under linearization errors, ensuring landmark observability despite partial fields of view, and synchronizing asynchronous data from wheel encoders, camera frames, and [Vicon](https://www.vicon.com/ "Vicon") ground-truth captures.
Moreover, AprilTag detection is constrained by lighting artifacts and pose ambiguity at shallow viewing angles, introducing non-Gaussian errors that the EKF must approximate linearly.
Moreover, tuning noise parameters presents the **classical tradeoff**: too little noise leads to overconfidence and divergence; too much noise leads to filter paralysis. Deployment exposes the systemic difference between simulation and physical experiments: real Duckiebots do not move with perfect kinematics, cameras suffer from radial distortion, and computation suffers from non-deterministic latency.
In SLAM, everything that can drift will drift, and the role of the filter is to drift more slowly than entropy.
##### Did this work spark your curiosity?
Check out the follow works on path planning with Duckietown:
- [Path Planning for Multi-Robot Navigation in Duckietown](https://duckietown.com/path-planning-for-multi-robot-navigation-in-duckietown/ "Path Planning for Multi-Robot Navigation in Duckietown")
- [Monocular Navigation in Duckietown Using LEDNet Architecture](https://duckietown.com/monocular-navigation-in-duckietown-using-lednet-architecture/ "Monocular Navigation in Duckietown Using LEDNet Architecture")
- [Goto-1: Planning with Dijkstra](https://duckietown.com/goto-1-planning-with-dijkstra/ "Goto-1: Planning with Dijkstra")
## Extended Kalman Filter (EKF) SLAM for Duckiebots: Authors
[  ](https://www.linkedin.com/in/amirhosseinzamani/?originalSubdomain=ca)
[AmirHossein Zamani](https://www.linkedin.com/in/amirhosseinzamani/?originalSubdomain=ca "AmirHossein Zamani") was a former Duckietown student, and currently, he is pursuing his Ph.D. in Computer Science at [Mila](https://mila.quebec/en "Mila") ([Quebec AI Institute](https://mila.quebec/en/directory/amirhossein-zamani "Quebec AI Institute")) and [Concordia University](https://www.concordia.ca/ "Concordia University"), Canada. He is also working as an AI Research Scientist Intern at [Autodesk](https://www.autodesk.com/ "Autodesk") in Montreal, Canada.
[  ](https://oestoleary.com/)
[Léonard Oest O’Leary](https://oestoleary.com/ "Léonard Oest O’Leary") was a former Duckietown student, and currently, he is pursuing his Master of Science in Computer Science at the [University of Montreal](https://www.umontreal.ca/en/ "University of Montreal"), Canada.
[  ](https://www.linkedin.com/in/kevin-lessard/?originalSubdomain=ca)
[Kevin Lessard](https://www.linkedin.com/in/kevin-lessard/?originalSubdomain=ca "Kevin Lessard") was a former Duckietown student, and currently, he is pursuing his Master of Science in Machine Learning at [Mila – Quebec AI Institute](https://mila.quebec/en "Mila - Quebec AI Institute") in Montreal, Canada.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Localization, Projects
**Tags:** navigation, path planning, project, student project
---
### [Duckiebot Localization with Sensor Fusion in Duckietown](https://duckietown.com/localization-with-sensor-fusion-in-duckietown/)
**Published:** July 8, 2025
**Author:** Duckietown Admin
**Excerpt:** This project explores localization in Duckietown using sensor fusion to estimate Duckiebot poses through visual tags and stitched camera inputs.
**Content:**
# Duckiebot Localization with Sensor Fusion in Duckietown
##### Project Resources
- **Objective**: Demonstrate accurate localization of Duckiebots using visual fiducial markers and sensor fusion in a controlled Duckietown environment.
- **Approach**: Use multi-camera sensor fusion to detect AprilTags and reconstruct a stitched top-down map for Duckiebot pose localization.
- **Authors**: Samuel Neumann, University of Alberta, Canada
[ University ](https://www.ualberta.ca/en/index.html)
[ Authors ](#authors)
[ Code ](https://github.com/samuelfneumann/CMPUT503)
[ Project ](https://samuelfneumann.github.io/posts/duckie_3/)
## Localization with Sensor Fusion in Duckietown - the objectives
The advantage of having multiple sensors on a Duckiebot is that the data provided can be combined to provide additional precision and reduce uncertainty in derived results. This process is generally referred to as sensor fusion, and a typical example is localization, i.e., the problem of finding the pose of the Duckiebot in time, with respect to some reference frame. And if the data is redundant? No problem, just discard it.
In this project, the objective is to implement sensor fusion-based localization and lane-following on a DB21 Duckiebot, integrating odometry (using data from wheel encoders) with visual [AprilTag](https://roboticsknowledgebase.com/wiki/sensing/apriltags/ "AprilTag") detection for improved positional accuracy.
This process addresses limitations of odometry, i.e., the open-loop reconstruction of the robots’ trajectory using only wheel encoder data in a mathematical approach known as “dead reckoning”, by incorporating AprilTags as global reference landmarks, thereby enhancing spatial awareness in environments where dead reckoning alone is insufficient.
Technical concepts include AprilTag-based localization, PID control for lane following, transform tree management in [ROS](https://docs.duckietown.com/daffy/instructor-manual/resources/slides/tools/02-ros.html "ROS") (tf2), and coordinate frame transformations for pose estimation.
[ Learn about odometry and april tags in Duckietown ](https://duckietown.com/educational-resources/#education-materials-modcon)
## Sensor fusion - visual project highlights
## The technical approach and challenges
This approach, at the technical level, involves:
- extending ROS-based packages to implement AprilTag detection using the dt-apriltags library,
- configuring static transformations for landmark localization in a unified world frame, and
- correcting odometry drift by broadcasting transforms from estimated AprilTag poses to the Duckiebot’s base frame.
A full PID controller was moreover implemented, with tunable gains for lateral and heading deviation, and derivative terms were conditionally initialized for stability.
Challenges included:
- remapping ROS topics for motor command propagation,
- resolving frame connectivity in tf trees,
- configuring accurate static transforms for AprilTag landmarks,
- debugging quaternion misrepresentation during pose updates, and
- correctly applying transform compositions using lookup\_transform\_full to compute odometry corrections.
![Camera and AprilTag coordinate frames used in localization and sensor fusion for pose estimation]()
AprilTag and Camera Frame Axes for Localization
![Duckiebot camera coordinate frame used in localization and sensor fusion for transforming AprilTag detections]()
Duckiebot Camera Frame Axes for Sensor Fusion
![Duckietown global coordinate system used in localization and sensor fusion for world-frame pose calculations]()
Duckietown Map Coordinate System for Global Localization
##### Looking for similar projects?
Check out the following works on path planning with Duckietown:
- [Path Planning for Multi-Robot Navigation in Duckietown](https://duckietown.com/path-planning-for-multi-robot-navigation-in-duckietown/ "Path Planning for Multi-Robot Navigation in Duckietown")
- [Adaptive Lane Following with Auto-Trim tuning](https://duckietown.com/adaptive-lane-following-with-automatic-trim-calibration/)
- [Deep Reinforcement Learning for Autonomous Lane Following ](https://duckietown.com/deep-reinforcement-learning-for-autonomous-lane-following/)
## Localization with Sensor Fusion in Duckietown: Authors
[  ](https://samuelfneumann.github.io/)
[Samuel Neumann](https://samuelfneumann.github.io/ "Samuel Neumann") is a Ph. D. student at the [University of Alberta](https://www.ualberta.ca/en/index.html "University of Alberta"), Canada.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://duckietown.com/guides/)
[ More Projects ](https://duckietown.com/best-robotics-and-ai-projects-for-learning-autonomy/)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Localization, Projects
**Tags:** perception, project, sensor fusion, student project
---
### [Intersection Navigation in Duckietown Using 3D Image Features](https://duckietown.com/intersection-navigation-in-duckietown-using-3d-image-feature/)
**Published:** December 23, 2024
**Author:** Duckietown Admin
**Excerpt:** Explore how 3D image features enhance Duckiebot intersection navigation in Duckietown, blending cutting-edge BEV tech with hands-on autonomous driving insights.
**Content:**
# Intersection Navigation in Duckietown Using 3D Image Features
##### Project Resources
- **Objective**: To evaluate the effectiveness of 3D-encoded image feature representations for intersection navigation in Duckietown using a Bird's Eye View (BEV) approach.
- **Approach**: Integrate the BEV encoding from MILE into the intersection navigation method proposed by Giles et al. (2019) and assess its performance in the Duckietown environment.
- **Authors**: Jasper Mulder
[ Report ](#thesis)
[ University ](https://www.uva.nl/en)
[ Authors ](#authors)
## Project highlights
Here is a visual tour of the authors’ work on implementing intersection navigation using 3D image features in Duckietown.
![Example showing camera input and its transformation into Bird's Eye View (BEV) space using a homographic matrix.]()
Figure 1. Camera Input to Bird's Eye View (BEV) Transformation.
![Classified stop line clusters in a Bird's Eye View (BEV) with colored dots representing predicted stop line locations and white dots indicating cluster centers.]()
Figure 2. Classified Stop Line Clusters in BEV.
![Three possible trajectories illustrated for navigating an intersection in a Bird's Eye View (BEV) representation.]()
Figure 3. Possible Intersection Navigation Trajectories.
![Camera input image used for generating Bird's Eye View (BEV) representations during intersection navigation.]()
Figure 4. Camera Input for BEV Generation in Intersection Navigation.
![Comparison of Bird's Eye View (BEV) representations from two methods, showing estimated stop lines with colored clusters and white dots indicating cluster centers.]()
Figure 5. Comparison of BEV Representations During Intersection Navigation.
![Comparison of MILE-generated BEVs from CARLA and Duckietown simulators, showing camera inputs and corresponding BEVs with non-drivable areas highlighted.]()
Figure 6. Comparison of MILE-Generated BEVs from CARLA and Duckietown Simulators.
## Intersection Navigation in Duckietown: Advancing with 3D Image Features
Intersection navigation in Duckietown using 3D image features is an approach intented to improve autonomous intersection navigation, enhancing decision-making and path planning in [complex Duckietown environments](https://docs.duckietown.com/daffy/opmanual-duckietown/appearance_specifications/tilemap/index.html "Duckietown tile maps"), i.e., made of several road loops and road intersections.
The traditional approach to intersection navigation in Duckietown is naive: (a) stop at the red line before the intersection, (b) read [Apriltag-equipped traffic signs](https://docs.duckietown.com/daffy/opmanual-duckietown/assembly/traffic_signs/index.html "Duckietown Traffic Signs") (providing information on the shape and coordination mechanism at intersections); (c) decide which direction to take; (d) coordinate with other vehicles at the intersection to avoid collisions; (e) navigate through the intersection. This last step is performed in an open-loop fashion, leveraging the known appearance specifications of intersections in Duckietown.
By incorporating 3D image features in the perception pipeline, extrapolated from the Duckietown road lines, Duckiebots can achieve a representation of their pose while crossing the intersection, closing, therefore, the loop and improving navigation accuracy, in addition to facilitating the development of new strategies for intersection navigation, such as real-time path optimization.
Combining 3D image features with methods, such as Bird’s Eye View (BEV) transformations allows for comprehensive representations of the intersection. The integration of these techniques improves the accuracy of stop line detection and obstacle avoidance contributes to advancing autonomous navigation algorithms and supports real-world deployment scenarios.
 An AI representation of Duckietown intersection navigation challenges
[ Teach and Learn Robot Autonomy with Duckietown ](https://duckietown.com/educational-resources/#autonomy)
## The method and the challenges of intersection navigation using 3D features
The thesis involves implementing the MILE model (Model-based Imitation LEarning for urban driving), trained on the [CARLA simulator](https://carla.org/ "CARLA autonomous driving simulator"), into the Duckietown environment to evaluate its performance in navigating unprotected intersections.
Experiments were conducted using the [Gym-Duckietown simulator](https://github.com/duckietown/gym-duckietown "Gym-Duckietown simulator"), where Duckiebots navigated a 4-way intersection across multiple trajectories. Metrics such as success rate, drivable area compliance, and ride comfort were used to assess performance.
The findings indicate that while the MILE model achieved state-of-the-art performance in the CARLA simulator, its generalization to the Duckietown environment without additional training was, as probably expected due to the sim2real gap, limited.
The BEVs generated by MILE were not sufficiently representative of the actual road surface in Duckietown, leading to suboptimal navigation performance. In contrast, the homographic BEV method, despite its assumption of a flat world plane, provided more accurate representations for intersection navigation in this context.
As for most approaches in robotics, there are limitation and tradeoffs to analyze.
Here are some technical challenges of the proposed approach:
- **Generalization across environments**: one of the challenges is ensuring that the 3D image feature representation generalizes well across different simulation environments, such as Duckietown and CARLA. The differences in scale, road structures, and dynamics between simulators can impact the performance of the navigation system.
- **Accuracy of BEV representations**: the transformation of camera images into Bird’s Eye View (BEV) representations has reduced accuracy, especially when dealing with low-resolution or distorted input data.
- **Real-time processing**: the integration of 3D image features for navigation requires substantial computational resources with respect to utilizing 2D features instead. Achieving near real-time processing speeds for navigation tasks such as intersection navigation, is challenging.
## Intersection Navigation in Duckietown Using 3D Image Feature: Full Report
## Intersection Navigation in Duckietown Using 3D Image Feature: Authors
[  ](https://www.linkedin.com/in/jasper-mulder-33977b201/)
[Jasper Mulder](https://www.linkedin.com/in/jasper-mulder-33977b201/) is currently working as a Junior Outdoor expert at [Bever](http://bever.nl/), Netherlands.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Perception, Projects
**Tags:** project, student project
---
### [Visual Obstacle Detection using Inverse Perspective Mapping](https://duckietown.com/visual-obstacle-detection-using-inverse-perspective-mapping/)
**Published:** January 17, 2025
**Author:** Duckietown Admin
**Excerpt:** This project develops a visual obstacle detection system in Duckietown using inverse perspective mapping to improve autonomous navigation accuracy.
**Content:**
# Visual Obstacle Detection using Inverse Perspective Mapping
##### Project Resources
- **Objective**: To develop a visual obstacle detection system for Duckiebots using inverse perspective mapping to improve navigation accuracy and safety.
- **Approach**: Employ inverse perspective mapping to transform monocular camera inputs into Bird’s Eye View representations, enabling reliable obstacle detection and classification.
- **Authors**: Julian Nubert, Niklas Funk, Fabio Meier, Fabrice Oehler
[ Report ](#thesis)
[ University ](https://ethz.ch/en.html)
[ Authors ](#authors)
## Project highlights
Here is a visual tour of the authors’ work on implementing visual obstacle detection in Duckietown.
![Monocular camera image capturing a Duckiebot's perspective of the road for visual obstacle detection.]()
Figure 1. Example Image from Monocular Camera.
![Bird’s Eye View image created from monocular camera input using inverse perspective mapping.]()
Figure 2. Image Transformed to Bird’s Eye View.
![Final detection output showing identified obstacles in Bird’s Eye View.]()
Figure 3. Final Detection Output.
![Cropped version of the monocular camera input, focusing on relevant road sections.]()
Figure 4. Cropped Image for Efficient Detection.
![Bird’s Eye View showing detected obstacle boxes overlayed on the road.]()
Figure 5. Display of Obstacle Boxes in Bird’s Eye View.
![Detected obstacle in Bird’s Eye View with position and radius annotations.]()
Figure 6. Position and Radius of Obstacle.
![Bird’s Eye View image classifying obstacles as dangerous or non-dangerous based on their position.]()
Figure 7. Dangerous vs. Non-Dangerous Obstacles.
![Search lines in Bird’s Eye View used to determine if white lane boundaries lie between the Duckiebot and an obstacle.]()
Figure 8. Search Lines for Lane Boundary Detection.
![Flowchart showing initial logic stages for obstacle handling during commissioning.]()
Figure 9. Initial Logic Stages for Commissioning.
![Definitions of variables in obstacle detection, as seen from the top view.]()
Figure 10. Top-View Variable Definitions.
![Geometry depicting the Duckiebot’s path and an obstacle's position in the lane.]()
Figure 11. Geometry of Scene and Obstacle Positioning.
![Diagram showing the software architecture for obstacle detection and avoidance in Duckietown.]()
Figure 12. Software Architecture Overview.
![Example of obstacle detection error caused by motion blur.]()
Figure 13. Motion Blur Impact on Obstacle Detection.
![Example of an adaptive bounding box conforming to lane curvature for improved obstacle detection.]()
Figure 14. Adaptive Bounding Box for Lane Curvature.
## Visual Obstacle Detection: objective and importance
This project aims to develop a visual obstacle detection system using inverse perspective mapping with the goal to enable autonomous systems to detect obstacles in real time using images from a monocular RGB camera. It focuses on identifying specific obstacles, such as yellow Duckies and orange cones, in Duckietown.
The system ensures safe navigation by avoiding obstacles within the vehicle’s lane or stopping when avoidance is not feasible. It does not utilize learning algorithms, prioritizing a hard-coded approach due to hardware constraints. The objective includes enhancing obstacle detection reliability under varying illumination and object properties.
It is intended to simulate realistic scenarios for autonomous driving systems. Key metrics of evaluation were selected to be detection accuracy, false positives, and missed obstacles under diverse conditions.
[ Teach and Learn Robot Autonomy with Duckietown ](https://duckietown.com/educational-resources/#autonomy)
## The method and the challenges visual obstacle detection using Inverse Perspective Mapping
The system processes images from a monocular RGB camera by applying inverse perspective mapping to generate a bird’s-eye view, assuming all pixels lie on the ground plane to simplify obstacle distortion detection. Obstacle detection involves [HSV color](https://en.wikipedia.org/wiki/HSL_and_HSV) filtering, image segmentation, and classification using eigenvalue analysis. The reaction strategies include trajectory planning or stopping based on the detected obstacle’s position and lane constraints.
Computational efficiency is a significant challenge due to the hardware limitations of Raspberry Pi, necessitating the avoidance of real-time re-computation of color corrections. Variability in lighting and motion blur impact detection reliability, while accurate calibration of camera parameters is essential for precise 3D obstacle localization. Integration of avoidance strategies faces additional challenges due to inaccuracies in pose estimation and trajectory planning.
## Visual Obstacle Detection using Inverse Perspective Mapping: Full Report
## Visual Obstacle Detection using Inverse Perspective Mapping: Authors
[  ](https://www.linkedin.com/in/juliannubert/?originalSubdomain=de)
[Julian Nubert](https://www.linkedin.com/in/juliannubert/?originalSubdomain=de) is currently a Research Assistant & Doctoral Candidate at the [Max Planck Institute for Intelligent Systems](https://is.mpg.de/social-policy), Germany.
[  ](https://www.linkedin.com/in/niklas-wilhelm-funk/?originalSubdomain=de)
[Niklas Funk](https://www.linkedin.com/in/niklas-wilhelm-funk/?originalSubdomain=de) is a PHD Graduate Student at [Technische Universität Darmstadt](https://www.tu-darmstadt.de/), Germany.
[  ](https://www.linkedin.com/in/fabiomeier/)
[Fabio Meier](https://www.linkedin.com/in/fabiomeier/) is currently working as the Head of Operational Data Intelligence at [Sensirion Connected Solutions](https://sensirion-connected.com/), Switzerland.
[  ](https://www.linkedin.com/in/fabriceoehler/)
[Fabrice Oehler](https://www.linkedin.com/in/fabriceoehler/) is working as a Software Engineer at [Sensirion](https://www.sensirion.com/), Switzerland.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Perception, Projects
**Tags:** project, student project
---
### [Autoduck: VLM-based Autonomous Navigation in Duckietown](https://duckietown.com/autoduck-vlm-autonomous-navigation-in-duckietown/)
**Published:** September 12, 2025
**Author:** Duckietown Admin
**Excerpt:** This project implements an autonomous navigation system on the Duckiebot DB21J in the Duckietown environment using vision-based control and VLMs.
**Content:**
# Autoduck: VLM-based Autonomous Navigation in Duckietown
##### Project Resources
- **Objective**: Develop an autonomous navigation system for Duckiebot DB21J in Duckietown using vision-based control and VLM integration.
- **Approach**: Implement calibrated perception, control, and FSM modules with ROS and integrate quantized Qwen 2.5 models on embedded hardware.
- **Authors**: Sahil Virani, Supratik Patel, Esmir Kico, Technical University of Munich
[ University ](https://www.tum.de/)
[ Report ](#report)
[ Code ](https://github.com/DuckietownTUM/GlitchieDuck)
## Dual-Mode Autonomous Navigation in Duckietown using VLM - the objectives
This project aims to implement an autonomous navigation system on the Duckiebot **DB21J** platform within Duckietown to enable vision-based control and decision-making using **VLM** (Vision Language Model).
The system integrates calibrated camera intrinsics and extrinsics, motor gain and trim calibration, ROS nodes for perception and control, AprilTag-based semantic localization, stop line detection, lane filter for lateral pose estimation, finite state machine for state transitions, PID controllers for velocity and steering regulation, and quantized Qwen 2.5 models for multimodal inference on embedded hardware.
The work establishes a **reproducible pipeline for benchmarking navigation algorithms**, enabling analysis of trade-offs between model size, inference latency, memory limits, communication overhead, and control cycle timing in real-time robotic systems.
[ Learn about odometry and april tags in Duckietown ](https://duckietown.com/educational-resources/#education-materials-modcon)
## VLM in Duckietown - visual project highlights
## The technical approach and challenges
This approach, at the technical level, involves:
The method integrates calibrated camera intrinsics and extrinsics for distortion correction and frame alignment, motor gain and trim calibration for odometry consistency, and ROS-based perception nodes for lane filtering, stop line detection, obstacle recognition, and AprilTag-based pose estimation. Control nodes implement PID regulators for velocity and steering, parameterized turning primitives, and synchronized execution through a finite state machine that coordinates lane following, intersection stopping, turning maneuvers, and recovery states. Sensor fusion combines camera streams, encoder feedback, and ToF measurements for robust decision inputs.
Quantized Qwen 2.5 vision-language models were deployed with llama.cpp, configured with reduced context window and batch size to match GPU memory limits. The models were evaluated for trajectory planning and visual reasoning tasks, with both 7B and 3B variants tested under quantization schemes. Integration required Docker containerization for portability and ROSBridge for monitoring and remote interaction.
Challenges included GPU memory capacity restricting larger model execution, inference latency exceeding 100 ms control cycle requirements, CUDA feature mismatches across builds, and instability in container runtimes on the NVIDIA Jetson Nano platform. These issues necessitated systematic parameter tuning of controllers, quantization of VLMs to GGUF formats, pruning strategies to reduce computation load, and hybrid offloading of visual reasoning to external compute nodes while maintaining low-level perception and control locally. Additional constraints involved balancing message-passing overhead in ROS, synchronization delays between perception and control nodes, and variability in inference reproducibility across different hardware builds.
![Autonomous navigation system using VLM Duckiebot DB21J assembly components]()
Duckiebot DB21J Assembly
![Autonomous navigation system using VLM Duckiebot DB21J data workflow]()
System Workflow Architecture
![Autonomous navigation system using VLM Duckiebot DB21J camera calibration patterns]()
Camera Calibration
![Autonomous navigation system using VLM Duckiebot DB21J Qwen 2.5 model latency accuracy comparison]()
VLM Model Performance
## Report and Presentation
##### Looking for similar projects?
Check out the following works on path planning with Duckietown:
- [Path Planning for Multi-Robot Navigation in Duckietown](https://duckietown.com/path-planning-for-multi-robot-navigation-in-duckietown/ "Path Planning for Multi-Robot Navigation in Duckietown")
- [Adaptive Lane Following with Auto-Trim tuning](https://duckietown.com/adaptive-lane-following-with-automatic-trim-calibration/)
- [Deep Reinforcement Learning for Autonomous Lane Following ](https://duckietown.com/deep-reinforcement-learning-for-autonomous-lane-following/)
## VLM in Duckietown: Authors
Sahil Virani is a student at [Technical University of Munich](https://www.tum.de/ "Technical University of Munich"), Germany.
[  ](https://www.linkedin.com/in/supratikpatel/)
[Suparatik Patel](https://www.linkedin.com/in/supratikpatel/ "Suparatik Patel") is a student at [Technical University of Munich](https://www.tum.de/ "Technical University of Munich"), Germany.
Esmir Kico is a student at [Technical University of Munich](https://www.tum.de/ "Technical University of Munich"), Germany.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://duckietown.com/guides/)
[ More Projects ](https://duckietown.com/best-robotics-and-ai-projects-for-learning-autonomy/)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Perception, Projects
**Tags:** perception, student project, Vision-based, VLM
---
### [Learning robot autonomy with Duckiedrones](https://duckietown.com/learning-robot-autonomy-with-duckiedrones/)
**Published:** November 12, 2025
**Author:** Duckietown Admin
**Excerpt:** Saif Chaudry, Computer Science student at the College of Charleston, South Carolina, tells us about his experience learning robot autonomy with Duckiedrones.
**Content:**
# Learning robot autonomy with Duckiedrones
Saif Chaudry, Computer Science student at the College of Charleston, South Carolina, tells us about his experience learning robot autonomy with Duckiedrones.
**Charleston, USA, November 2025:** Saif Chaudry, junior majoring in Computer Science with a minor in Data Science at the College of Charleston, South Carolina, talks to us about his experience learning robot autonomy using Duckiedrones and developing an autonomous inventory system.
##### Quick links
- [ Saif Chaudry ](https://www.linkedin.com/in/chaudhrysa/)
- [ College of Charleston ](https://charleston.edu/)
- [ The Drone Lab ](https://charleston.edu/compsci/experiential-learning/cs-research-labs.php)
- [ Get a Duckiedrone DD24-B ](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24)
## Learning robot autonomy at the College of Charleston
##### Thank you for your time and for being here. Could you please introduce yourself and tell us what you do?
Sure. My name is Saif, and I currently attend the College of Charleston in South Carolina, USA. I’m in the Honors College and I’m a junior majoring in Computer Science, with a minor in Data Science.
I started doing research at the college’s Drone Lab in the summer of 2024. Back then, we worked with DJI drones, mainly the DJI Tello and other models. It was a great hands-on experience learning robot autonomy and how to make drones scan barcodes and navigate autonomously.
This past summer, though, we switched over to the Duckiedrone model DD24-B ([review DD24-B Duckiedrone documentation](https://docs.duckietown.com/ente/opmanual-dd24/intro.html), or [get a DD24-B](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24 "Duckietown project Shop DD24-B Duckiedrone")), which turned out to be a great experience. It was intuitive to use and worked really well for our research.
##### That’s great. You already mentioned how you got involved with **Duckietown** and the Duckiedrones. Was there a specific reason you switched to them?
When we started with the DJI Tello drone, it was good for learning robot autonomy at a basic level. Later, we moved to more advanced models like the DJI Mavic Air 2 and the DJI Mini 3 Pro. But we ran into issues, the documentation was mostly in Chinese, and the SDKs were outdated, which made development difficult.
One of my professors, [Dr. Mia Y. Wang](https://www.linkedin.com/in/mia-wang-cofc/), had another student who recommended the Duckiedrone. He thought it was a cool product to learn about robot autonomy, so Dr. Wang ordered a few units. That’s how we started using them this past summer.
##### And what was your experience like with the Duckiedrone? You mentioned it was easy to use, did you manage to achieve your project goals?
Our goal this summer was to develop an autonomous inventory system using drones. I worked on the project with my research partner, [Samuel Eubank](https://www.linkedin.com/in/samuel-eubank-324a33108/ "Samuel Eubank"). Sammy built the drone physically while I worked remotely on the software side.
I set up the SD card, connected it to the internet, and got it communicating with my computer. We had some issues with the flight and infrared sensors, but I was able to fix them. Eventually, the drone started flying properly.
Now, this semester, we’re continuing the project, specifically focusing on getting the drone to scan barcodes.
##### Did you find the available Duckietown documentation helpful?
Yes, definitely. The [Duckiedrone DD24 (daffy) documentation](https://docs.duckietown.com/daffy/opmanual-duckiedrone/intro.html "Duckietown Duckiedrone Documentation (daffy)") was very straightforward, it clearly explained how to get the drone connected to the internet and the computer, and the terminal commands were well documented.
I also joined the Duckietown Slack community and the Stack Overflow discussions. The community is very active, and it really helped me learn more about robot autonomy and troubleshoot issues. I even saw students from other universities helping each other out.
##### That’s great to hear. What are your next steps with this project?
We’re continuing the autonomous inventory project this semester. We had some problems with the college Wi-Fi, there were firewalls blocking access to the Raspberry Pi on the drone. But we managed to solve that using a VPN.
Now we’re working on another issue: the drone doesn’t fly high enough off the ground. We think it’s related to the maximum throttle settings, and we’re getting help from people on Slack to fix it. That’s our next goal.
> I joined the Duckietown community and the Duckietown Archives. The community is very active, and it really helped me troubleshoot issues. I even saw students from other universities helping each other out.
>
> Saif Chaudry
##### That sounds exciting. Do you want to add anything about your future goals?
I’ve always been passionate about learning robot autonomy, especially self-driving cars. I find what companies like Waymo and Tesla are doing fascinating.
This autonomous inventory project is helping me learn how to make systems navigate autonomously, from point A to point B, planning paths, and operating indoors. In the future, I’d love to work in the field of autonomous systems and help develop technologies that make things more self-sufficient.
### Learn more about Duckietown
[Duckietown](https://duckietown.com/) enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
**Categories:** People
**Tags:** autonomous drones, autonomous systems, duckiedrone, robotic project, student project
---
### [University of Nevada: Professor Lei Yang uses Duckietown based RET](https://duckietown.com/university-of-nevada-professor-lei-yang-uses-duckietown-based-ret/)
**Published:** March 14, 2023
**Author:** Ivano Marocchi
**Content:**
**University of Reno, March 7, 2023**: Lei Yang, Associate Professor in the Department of Computer Science and Engineering of the University of Nevada in Reno shares with us his experience conducting a program called Research Experience for Teachers (RET) focused on “Integrating Big Data into Robotics”.
##### Quick links
- [ University of Nevada ](https://www.unr.edu/)
- [ Integrating Big Data into Robotics: Research Experience for Teachers ](https://www.unr.edu/nevada-today/news/2022/duckietown)
- [ Prof. Lei Yang homepage ](https://www.linkedin.com/in/lei-yang-63239921/)
## Professor Lei Yang tells us about conducting a Duckietown based Research Experience for Teachers
Professor Lei Yang shares with us his relationship with Duckietown and how it performed used in a K-12 teachers research experience led by the Computer Science and Engineering department of the University of Nevada in Reno.
**Good morning Professor and thank you for finding the time to speak to me.**
Good morning, thank you.
**How did you come across Duckietown the first time? When did you discover it?**
Well, we needed a specific platform for our project, and a collaborator from Europe told us about this platform he was very familiar with. He let us know that it was a great platform, and that we should have a look at it. We accepted and ended up using Duckietown and suiting it to our project.
**Could you tell us more about this project?**
For three years now at the University of Nevada, Reno’s Computer Science and Engineering department (CSE) we’ve been conducting a program called Research Experience for Teachers (RET) focused on “Integrating Big Data into Robotics”. It’s a six-week course, through which participants can gain hands-on robotics experience that can be later applied in classrooms, in a fun way. The main idea is trying to provide a research experience to K-12 teachers. That’s why we proposed the idea of using Duckietown to teach K-12 teachers. We asked ourselves. what is the state of the art in terms of data analytics, machine learning? I think Duckietown is a very good education platform for teachers. We make use of the very good materials provided by Duckietown and I’m very satisfied with its implementation.
We purchased a Duckietown set for each participant and let them bring the hardware back to their school. Some teachers started their very own robotics clubs! They basically utilize that as an additional platform for their students.
**What would you say are the characteristics of Duckietown that make it useful for you?**
Our RET program involves all K-12 teachers, and one of the main goals of our program is to work with these teachers to develop curriculum modules suitable for their students. We have teachers from different levels, but we find that actually middle school and high school teachers are kind of more suited for this program. Duckietown is freely available and includes curricula that can be adapted for all levels of education. It is tangible, it is accessible, and looks fun!
> "I think Duckietown is a very good education platform for teachers. We make use of the very good materials provided by Duckietown and I’m very satisfied with its implementation."
>
> Prof. Lei Yang
It’s also easy to deal with. We can find all the materials online, and it is hands-on as I already mentioned. People like hands on activities, it’s good for kids. The duckies also serve to present robotics as less intimidating, making it easier to teach the harder, underlying concepts. I think that’s very nice: it can be used to teach optimization, control theory, these are fundamental things. I think this is a platform that can suit people with different levels of background and also an easy way to start one’s journey into robotics.
**What does the future hold?**
I think we’ve done a great job this year, and the teachers liked our project. I can see a significant improvement compared to the first year. This is a three-year project, so this year was the last for the program. After the program expired, we submitted another proposal to continue utilizing Duckietown, and to integrate new things like blockchain technology and other new ideas into this program so hopefully we’ll be using this platform in the future as well.
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
[ Find out more use cases ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** Blog, News, People
---
### [Autonomous Navigation and Parking in Duckietown](https://duckietown.com/autonomous-navigation-and-parking-in-duckietown/)
**Published:** May 18, 2025
**Author:** Duckietown Admin
**Excerpt:** This project uses PID control, AprilTag-based turns, dead reckoning, and visual servoing to enable autonomous navigation and parking in Duckietown.
**Content:**
# Autonomous Navigation and Parking in Duckietown
##### Project Resources
- **Objective**: Achieve autonomous navigation and parking in an intersection-equipped Duckietown city.
- **Approach**: Utilizing an FSM to transition between PID control-based lane following, intersection navigation, and parking using visual fiducial markers (AprilTags), color masking, and dead reckoning.
- **Authors**: Eric Khumbata, Cameron Hildebrandt, Jasper Eng from the University of Alberta.
- **Course**: This project is part of the CMPUT 412 Experimental Mobile Robotics course at the University of Alberta under Prof. Matthew E. Taylor.
[ University ](https://www.ualberta.ca/en/index.html)
[ Authors ](#authors)
[ Code ](https://github.com/ekhumbata/Adventures-in-Duckietown)
[ Course ](https://www.ualberta.ca/en/computing-science/undergraduate-studies/course-directory/courses/experimental-mobile-robotics.html)
## Project highlights
> Static parameters in a dynamic environment are pre-programmed failure points.
![Autonomous Navigation and Parking in Duckietown using pid control]()
Figure 1. Lane Following with AprilTag Decision Logic
![Autonomous Navigation and Parking in Duckietown using pid control]()
Figure 2. Obstacle and Pedestrian Detection with Color Masking
![Autonomous Navigation and Parking in Duckietown using pid control]()
Figure 3. Visual Parking Using AprilTags
[](https://duckietown.com/wp-content/uploads/2025/05/Lane-Following-with-AprilTag-Decision-Logic-png.avif) Figure 1. Lane Following with AprilTag Decision Logic [](https://duckietown.com/wp-content/uploads/2025/05/Obstacle-and-Pedestrian-Detection-with-Color-Masking-png.avif) Figure 2. Obstacle and Pedestrian Detection with Color Masking [](https://duckietown.com/wp-content/uploads/2025/05/Visual-Parking-Using-AprilTags-png.avif) Figure 3. Visual Parking Using AprilTags
## Autonomous Navigation and Parking in Duckietown: the objectives
This includes the development of a **closed-loop PID control** mechanism for continuous lane following, the use of **AprilTag detection** for intersection decision-making, and a state-driven behavior architecture to transition between tasks such as stopping, turning, and parking.
The system uses wheel encoder data for **dead-reckoning**-based motion execution in the absence of visual cues, and applies **HSV-based color segmentation** to detect and respond to static and dynamic obstacles. Visual servoing is used for parking alignment based on AprilTag localization. The control logic is modular and supports parameter tuning for hardware variability, with temporal filtering to suppress redundant detections and ensure stability.
[ Learn robot path planning with Duckietown ](https://duckietown.com/educational-resources/#planning)
## The challenges and approach
The key technical issues included inconsistent AprilTag detection due to motion blur and multiple redundant detections, which were mitigated using temporal filtering. PID control was used for continuous lane following, with dead reckoning based on wheel encoder data for intersection traversal when visual input was unreliable.
Obstacle detection and stopping mechanisms used HSV-based color segmentation to identify static and dynamic objects in the environment. In the parking stage, AprilTag-based localization and visual servoing were used to achieve stall alignment. The system was modular, with state-driven control logic managing transitions between lane following, intersection handling, obstacle detection, and parking.
##### Did this work spark your curiosity?
Check out the following works on path planning with Duckietown:
- [Path Planning for Multi-Robot Navigation in Duckietown](https://duckietown.com/path-planning-for-multi-robot-navigation-in-duckietown/ "Path Planning for Multi-Robot Navigation in Duckietown")
- [Monocular Navigation in Duckietown Using LEDNet Architecture](https://duckietown.com/monocular-navigation-in-duckietown-using-lednet-architecture/ "Monocular Navigation in Duckietown Using LEDNet Architecture")
- [Goto-1: Planning with Dijkstra](https://duckietown.com/goto-1-planning-with-dijkstra/ "Goto-1: Planning with Dijkstra")
## Autonomous Navigation and Parking in Duckietown: Authors
[  ](https://www.linkedin.com/in/eric-khumbata/?originalSubdomain=ca)
[Eric Khumbata](https://www.linkedin.com/in/eric-khumbata/?originalSubdomain=ca "Eric Khumbata") is working as a Computer Engineer at [TELUS](https://www.linkedin.com/company/telus/ "TELUS profile on LinkedIn"), a Canadian telecommunications company.
[  ](https://www.linkedin.com/in/jasper-eng/)
[Jasper Eng](https://www.linkedin.com/in/jasper-eng/ "Jasper Eng") is currently working as a Summer Research Intern at the BLINC Lab.
[  ](https://www.linkedin.com/in/kevin-lessard/?originalSubdomain=ca)
[Cameron Hildebrandt](https://www.linkedin.com/in/hldt/?originalSubdomain=ca "Cameron Hildebrandt") is currently working as a Fullstack Developer at [Bitcoin Well](https://bitcoinwell.com/ "Bitcoin Well"), Canada.
### Learn more
Duckietown is a capable, affordable, and reliable platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Projects
**Tags:** navigation, path planning, project, student project
---
### [Tor Vergata University and Duckietown deliver hands-on control systems workshop at EU Maker Faire Rome 2025](https://duckietown.com/tor-vergata-university-and-duckietown-partner-to-deliver-a-hands-on-control-systems-workshop-at-the-eu-maker-faire-rome-2025-2/)
**Published:** October 15, 2025
**Author:** Duckietown Admin
**Excerpt:** Tor Vergata University delivers a hands-on educational workshop on control systems at the EU Maker Faire Rome 2025 in partnership with Duckietown.
**Content:**
# Tor Vergata University and Duckietown deliver hands-on control systems workshop at EU Maker Faire Rome 2025
Tor Vergata University, Rome, Italy, delivered a hands-on educational workshop on control systems at the European Maker Faire 2025 that took place in Rome, Oct. 17-19, 2025, in partnership with Duckietown.
- [ European Maker Faire 2025 ](https://makerfairerome.eu/en/)
- [ Tor Vergata University ](https://web.uniroma2.it/)
- [ Dipartimento di Ingegneria Civile e Ingegneria Informatica Universita di Tor Vergata ](https://dicii.uniroma2.it/)
- [ Introduction to automatic control with self-driving cars workshop ](https://makerfairerome.eu/en/events-2025/?evento=hands-on-introduction-to-control-systems-the-hidden-technology-in-robotics&id=2191&giorno=17)
[ Maker Faire 2025 ](https://makerfairerome.eu/en/)
## Hands-on workshop: Introduction to automatic control with self-driving cars
Tor Vergata University, Rome’s second University, in partnership with Duckietown, has delivered a workshop titled “Introduction to Automatic Control with Self-Driving Duckiebots” in occasion of the EU Maker Faire in Rome, which hosted nearly 50000 visitors over the span of three days.
The objective of the workshop was to introduce participants to the fundamental principles of control system engineering and vehicle autonomy, by using real and simulated Duckiebots, and investigating the real-world impact of “details” such as controller tuning.
In this workshop, tuned for makers, educators, and learners, participants have:
- Learnt what a robot is and what all robots have in common;
- Understood the role of feedback and control systems in vehicle autonomy, as well as other everyday technologies;
- Explored sensors, actuators, and the perception pipeline of Duckiebots;
- Tuned a PID controller on simulated and physical Duckiebots.
## Learning automatic control: Who, where and when
The workshop, led by Professor Mario Sassano from the Dipartimento di Ingegneria Civile e Informatica of the Tor Vergata University, took place in three sessions at the European Maker Faire 2025 in Rome. Shima Akbari, Giorgio Manca and Davide Iafrate provided precious assistance:
**Friday, October 17, 2025: from 13:00 to 14:30 CET, Room 2 Make Lab (Area A)**
**Saturday, October 18, 2025: from 12.30 to 14:00 CET, Room 2 Make Lab (Area A)**
**Sunday, October 19, 2025: form 13:00 to 16:00, Room 8 (Area J)
[ Maker Faire 2025 ](https://makerfairerome.eu/en/)
#### Control Systems Workshop Speakers
Prof. Mario Sassano is Engineering Professor at the [University of Rome Tor Vergata](https://web.uniroma2.it/), Italy.
[Shima Akbari](https://www.linkedin.com/in/shima-a-3115391a0/ "Shima Akbari") is a Ph. D. student at Italian National Program in Autonomous Systems at the [University of Rome Tor Vergata](https://web.uniroma2.it/home "University of Rome Tor Vergata"), Italy.
[Giorgio Manca](https://www.linkedin.com/in/giorgio-manca-1434bb255/edit/forms/next-action/after-connect-update-profile/) is a Ph. D. Student in the DAuSy program at the [University of Rome Tor Vergata](https://web.uniroma2.it/), Italy.
[Jacopo Tani](https://www.linkedin.com/in/jacopo-tani/?originalSubdomain=ch), Ph. D. is cofounder, President and CEO of [Duckietown](https://duckietown.com/).
Prof. [Liam Paull](https://www.linkedin.com/in/liam-paull-83a5442b/?originalSubdomain=ca) is Associate Professor at [Universitè de Montréal](https://www.linkedin.com/school/universite-de-montreal/posts/?feedView=all), CTO and cofounder at Duckietown.
[Davide Iafrate](https://www.linkedin.com/in/davide-iafrate-b1a4011bb/) is a Robotics Engineer at [Duckietown](https://duckietown.com/).
### About Duckietown
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
**Categories:** Events, News
**Tags:** autonomous systems, control systems, feedback control, Maker Faire, robotics, Tor Vergata University, workshop
---
### [Duckietown at the European Maker Faire 2025 - Rome](https://duckietown.com/european-maker-faire-2025/)
**Published:** October 14, 2025
**Author:** Duckietown Admin
**Excerpt:** Duckietown goes to the European Maker Faire in Rome, where makers, innovators, and creatives from all over the world showcase their projects.
**Content:**
# Duckietown at the European Maker Faire 2025 – Rome
Duckietown went to the European Maker Faire 2025 in Rome, a place where makers, innovators, and creatives from all over the world showcase
projects in electronics, artificial intelligence, robotics, virtual and
augmented reality, gaming, music, art, education, and much more.
- [ European Maker Faire 2025 ](https://makerfairerome.eu/en/)
## Duckietown at the Maker Faire
**Maker Faire Rome – The European Edition** is an annual event, open to visitors, dedicated to innovation, technology, and creativity. It brings together innovators, makers, and enthusiasts from all over Europe. In addition to showcasing projects and inventions, it offers workshops, conferences, and labs to acquire technical skills and stimulate collaboration.
It attracts students, startups, companies, and government entities, fostering idea exchange and technological evolution. It has become a reference point for the European innovators community, highlighting Italy as a center of innovation and creativity.
Duckietown went to Rome from the 17th to the 19th of October, to showcase our robots, meet enthusiasts and other exhibitors, and talk about robotics and robot autonomy. And what a ride it has been! Here below are some photos we took at the event.
And for those of you who could not get a chance to talk to us at the event and get our contact, don’t forget to sign up to our new [Self-Driving Cars with Duckietown Massive Online Open Course](https://duckietown.com/self-driving-cars-with-duckietown-mooc/)!
[ Learn more about the Maker Faire ](https://makerfairerome.eu/en/what-is-maker-faire/)
### About Duckietown
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
**Categories:** Events
**Tags:** Maker Faire, robotics
---
### [Visual Control for Autonomous Navigation in Duckietown](https://duckietown.com/visual-control-for-autonomous-navigation-in-duckietown/)
**Published:** September 12, 2025
**Author:** Duckietown Admin
**Excerpt:** Visual control enables autonomous navigation of mobile robots in Duckietown by extracting lane features and computing steering commands from camera data.
**Content:**
##### General Information
- **Title**: Visual Urban Navigation for Mobile Robots: Implementation in the Duckietown Environment
- **Authors**: Shima Akbari, Nima Akbari, Giuseppe Oriolo, Sergio Galeani
- **Institution**: Università degli Studi di Roma Tor Vergata, Italy
- **Citation**: S. Akbari, N. Akbari, G. Oriolo and S. Galeani, "Visual Urban Navigation for Mobile Robots: Implementation in the Duckietown Environment," 2025 International Conference on Control, Automation and Diagnosis (ICCAD), Barcelona, Spain, 2025, pp. 1-6, doi: 10.1109/ICCAD64771.2025.11099311.
[ Paper ](https://ieeexplore.ieee.org/abstract/document/11099311)
[ Authors ](#authors)
[ Institution ](https://web.uniroma2.it/en)
# Visual Control for Autonomous Navigation in Duckietown
This research presents a visual control framework for in Duckietown using only onboard camera feedback for autonomous navigation. The system models the [Duckiebot](https://get.duckietown.com/products/duckiebot-db21 "Duckiebot") as a unicycle with constant driving velocity and uses steering velocity as the control input. Virtual guidelines are extracted from the lane boundaries to compute two visual features: the middle point and the vanishing point on the image plane.
The controller drives these features to the image center using a mathematically derived control law. The visual features are obtained from the camera feed using a multi-stage image processing pipeline implemented in OpenCV. The pipeline includes frame denoising, grayscale conversion, edge detection using the Canny edge detection algorithm, region of interest masking, and line detection via the Probabilistic Hough Line Transform. This setup provides robust detection of the white and yellow lane markings under varying conditions.
A scenario-driven transition system detects red lines marking intersections and activates artificial guidelines to execute controlled turns. The visual control implementation runs as a single ROS node following a publisher-subscriber architecture, deployed both in the Duckietown Simulator ([gym](https://github.com/duckietown/gym-duckietown "gym")) and in Duckietown.
![Visual control scheme for autonomous navigation in Duckietown]()
Figure 1. General Scheme of Robot Navigation
![Visual control environment in Duckietown]()
Figure 2. Duckietown Real vs Simulated Setup
![Visual control coordinate frames in Duckietown]()
Figure 3. Coordinate Frames and Virtual Guidelines
![Visual control image steps in Duckietown]()
Figure 4. Image Processing Steps Verification
![Visual control processing pipeline in Duckietown]()
Figure 5. Complete Image Processing Workflow
![Visual control turn timing in Duckietown]()
Figure 6. Turn Timing Schematics
![Visual control artificial guidelines in Duckietown]()
Figure 7. Artificial Guidelines for Turning
![Visual control lane centering in Duckietown]()
Figure 8. Lane Centering Point Convergence
![Visual control steering velocity in Duckietown]()
Figure 9. Velocity Convergence Graph
![Visual control turns evolution in Duckietown]()
Figure 10. Consecutive Turns Point Evolution
![Visual control turns velocity in Duckietown]()
Figure 11. Consecutive Turns Velocity Evolution
![Visual control camera workflow in Duckietown]()
Figure 12. Live Camera Image Processing
![Visual control lane experiment in Duckietown]()
Figure 13. Experimental Lane Centering
![Visual control experimental velocity in Duckietown]()
Figure 14. Experimental Velocity Evolution (Centering)
![Visual control turning experiment in Duckietown]()
Figure 15. Experimental Turning Trials
![Visual control right turn in Duckietown]()
Figure 16. Experimental Right Turn Velocity
![Visual control left turn in Duckietown]()
Figure 17. Experimental Left Turn Velocity
[ Learn about control systems with Duckietown ](https://duckietown.com/educational-resources/)
## Highlights - Visual Control for Autonomous Navigation in Duckietown
Here is a visual tour of the implementation of visual control for autonomous navigation by the authors. For all the details, check out the [full paper](#links "Post resources").
[](https://duckietown.com/wp-content/uploads/2025/09/Figure-1.-General-Scheme-of-Robot-Navigation-png.avif) Figure 1. General Scheme of Robot Navigation [](https://duckietown.com/wp-content/uploads/2025/09/Figure-2.-Duckietown-Real-vs-Simulated-Setup.avif) Figure 2. Duckietown Real vs Simulated Setup
[](https://duckietown.com/wp-content/uploads/2025/09/Figure-3.-Coordinate-Frames-and-Virtual-Guidelines-png.avif) Figure 3. Coordinate Frames and Virtual Guidelines [](https://duckietown.com/wp-content/uploads/2025/09/Figure-4.-Image-Processing-Steps-Verification.avif) Figure 4. Image Processing Steps Verification
[](https://duckietown.com/wp-content/uploads/2025/09/Figure-5.-Complete-Image-Processing-Workflow-png.avif) Figure 5. Complete Image Processing Workflow [](https://duckietown.com/wp-content/uploads/2025/09/Figure-6.-Turn-Timing-Schematics-png.avif) Figure 6. Turn Timing Schematics
[](https://duckietown.com/wp-content/uploads/2025/09/Figure-7.-Artificial-Guidelines-for-Turning-png.avif) Figure 7. Artificial Guidelines for Turning [](https://duckietown.com/wp-content/uploads/2025/09/Figure-8.-Lane-Centering-Point-Convergence-png.avif) Figure 8. Lane Centering Point Convergence
[](https://duckietown.com/wp-content/uploads/2025/09/Figure-9.-Velocity-Convergence-Graph-png.avif) Figure 9. Velocity Convergence Graph [](https://duckietown.com/wp-content/uploads/2025/09/Figure-10.-Consecutive-Turns-Point-Evolution-png.avif) Figure 10. Consecutive Turns Point Evolution
[](https://duckietown.com/wp-content/uploads/2025/09/Figure-11.-Consecutive-Turns-Velocity-Evolution-png.avif) Figure 11. Consecutive Turns Velocity Evolution [](https://duckietown.com/wp-content/uploads/2025/09/Figure-12.-Live-Camera-Image-Processing.avif) Figure 12. Live Camera Image Processing
[](https://duckietown.com/wp-content/uploads/2025/09/Figure-13.-Experimental-Lane-Centering.avif) Figure 13. Experimental Lane Centering [](https://duckietown.com/wp-content/uploads/2025/09/Figure-14.-Experimental-Velocity-Evolution-Centering-png.avif) Figure 14. Experimental Velocity Evolution (Centering)
[](https://duckietown.com/wp-content/uploads/2025/09/Figure-15.-Experimental-Turning-Trials.avif) Figure 15. Experimental Turning Trials [](https://duckietown.com/wp-content/uploads/2025/09/Figure-16.-Experimental-Right-Turn-Velocity-png.avif) Figure 16. Experimental Right Turn Velocity
[](https://duckietown.com/wp-content/uploads/2025/09/Figure-17.-Experimental-Left-Turn-Velocity.avif) Figure 17. Experimental Left Turn Velocity
## Abstract
Here is the abstract of the work, directly in the words of the authors:
This paper presents a vision-based control framework for the autonomous navigation of wheeled mobile robots in city-like environments, including both straight roads and turns. The approach leverages Computer Vision techniques and OpenCV to extract lane line features and utilizes a previously established control law to compute the necessary steering commands.
The proposed method enables the robot to accurately follow the lanes and seamlessly handle complex maneuvers such as consecutive turns. The framework has been rigorously validated through extensive simulations and real-world experiments using physical robots equipped with the ROS framework. Experimental evaluations were conducted at the DIAG Robotics Lab at Sapienza University of Rome, Italy, demonstrating the practicality of the proposed solution in realistic settings.
This work bridges the gap between theoretical control strategies and their practical application, offering insights into vision-based navigation systems for autonomous robotics. A video demonstration of the experiments is available at .
## Conclusion - Visual Control for Autonomous Navigation in Duckietown
Here is the conclusion according to the authors of this paper:
This paper proposed a vision-based control framework for lane-following tasks in wheeled mobile robots, validated through both simulations and real-world experiments. The approach effectively maintains the robot position at the center of lanes and enables safe left and right turns by relying solely on visual feedback from onboard camera, without requiring external localization systems or pre-mapped environments.
The system’s modular design and simplicity allow for seamless integration with other robotic systems, making it versatile for diverse urban navigation scenarios. Future research will focus on enhancing the framework to handle complex scenarios, such as autonomous lane corrections, and incorporating obstacle detection and avoidance mechanisms for improved performance in dynamic, real-world environments.
These advancements will expand the applicability of the proposed method, confirming its potential as a robust solution for autonomous navigation.
##### Did this work spark your curiosity?
Check out the following works on vehicle autonomy with Duckietown:
- - [Adapting World Models with Latent-State Dynamics Residuals](https://duckietown.com/adapting-world-models-with-latent-state-dynamics-residuals/)
- [Semantic Image Segmentation Methods in the Duckietown Project](https://duckietown.com/vision-based-drl-autonomous-driving-agent-with-sim2real-transfer-li-okhrin/ "Vision-Based DRL Autonomous Driving Agent with Sim2Real Transfer")
- [Visual Monitoring of Swarms of Industrial Robots](https://duckietown.com/visual-monitoring-of-automated-guided-vehicles-in-duckietown/)
- [Deep Reinforcement and Transfer Learning for Robot Autonomy](https://duckietown.com/deep-reinforcement-and-transfer-learning-for-robot-autonomy/ "Deep Reinforcement and Transfer Learning for Robot Autonomy")
- [Enhancing Visual Domain Randomization with Real Images for Sim-to-Real Transfer](https://duckietown.com/enhancing-visual-domain-randomization-for-sim2real-transfer/ "Enhancing Visual Domain Randomization with Real Images for Sim-to-Real Transfer")
#### Project Authors
[  ](https://www.linkedin.com/in/shima-a-3115391a0/)
[Shima Akbari](https://www.linkedin.com/in/shima-a-3115391a0/ "Shima Akbari") is a PhD student at Italian National Program in Autonomous Systems at the [University of Rome Tor Vergata](https://web.uniroma2.it/home "University of Rome Tor Vergata"), Italy.
[  ](https://www.linkedin.com/in/nima-akbari-63995a197/?originalSubdomain=ch)
[Nima Akbari](https://www.linkedin.com/in/nima-akbari-63995a197/?originalSubdomain=ch "Nima Akbari") is a PhD student at [Basel University of Switzerland](http://www.unibas.ch/ "Basel University of Switzerland") in privacy technologies for the Internet of Things.
[  ](https://www.linkedin.com/in/giuseppe-oriolo-17a969162/?originalSubdomain=it)
[Giuseppe Oriolo](https://www.linkedin.com/in/giuseppe-oriolo-17a969162/?originalSubdomain=it "Giuseppe Oriolo") is a Full Professor of Automatic Control and Robotics at [Sapienza University of Rome](http://www.uniroma1.it/ "Sapienza University of Rome").
[  ](https://scholar.google.com/citations?user=tk2snjYAAAAJ&hl=en)
[Sergio Galeani](https://scholar.google.com/citations?user=tk2snjYAAAAJ&hl=en "Sergio Galeani") is a full professor at the [University of Rome Tor Vergata](https://web.uniroma2.it/home "University of Rome Tor Vergata"), Italy.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** Research
**Tags:** CV
---
### [New Software Release - Ente Changelog](https://duckietown.com/duckietown-ente-release-changelog/)
**Published:** August 31, 2025
**Author:** Duckietown Admin
**Excerpt:** Duckietown's new software release (Ente) introduces virtual robots, the Duckiematrix, DTPS, refactored core, new robot apps, for ever more joyful learning.
**Content:**
# New Software Release – Ente Changelog
The Duckietown platform has been evolving since its creation back at MIT in 2016. The main code base has undergone four major revisions, with the current release named `daffy` (`d`: fourth letter of the alphabet).
We are now happy to announce the new major Duckietown software release: `ente`!
- [ Duckietown Codebase ](https://github.com/duckietown)
- [ The Duckietown (Ente) Manual ](https://docs.duckietown.com/ente/duckietown-manual/welcome-to-the-duckietown-manual.html)
- [ Other Ente technical documentation ](https://docs.duckietown.com/ente/)
- [ Get started with the Duckiematrix ](https://docs.duckietown.com/ente/duckietown-manual/50-duckiematrix/introduction-to-the-duckiematrix-virtual-environment.html)
- [ Get a Duckietown Robot ](https://get.duckietown.com/)
## Why ente?
First things first: why is it called `ente`?
Among the various meanings of this word in different languages, `Ente` is the German word for “duck”. We chose this name as a tribute of Duckietown to ETH Zürich, and the German-speaking part of Switzerland, for their influence on Duckietown’s evolution over the last years.
But why did we need `ente`?
We built `ente` to streamline the code base, especially the autonomy code running on Duckietown robots, make the development process quicker and more efficient, and to prime the platform for easier updates, maintenance, and future improvements.
The Duckietown codebase had evolved, historically, from a [classroom experience](https://duckietown.com/history/ "A brief history of Duckietown"), resulting in a improvable autonomy stack. The `ente` initiative grew to include infrastructural upgrades, e.g., the introduction of the Duckietown Postal System (DTPS), to better support reproducible robotics learning experiences in light of new developments in the fields of robotics and AI, e.g., the release of ROS2.
## What is new in ente?
Here is a non-exhaustive list of changes introduced by ente into Duckietown.
##### The Duckiematrix virtual environment
With ente comes the Duckiematrix, a photorealistic Unity-based virtual environment supporting virtual Duckietown robots.
The Duckiematrix allows simulating the physics and aesthetics of a physical Duckietown environment, as well as the sensing and acting capabilities of virtual Duckietown robots within that environment.
The Duckiematrix is programmable, lightweight, ROS compatible, and supports “multiplayer” features, where multiple learners can join the same city with their Duckiebots and learn & practice together.
[ Getting started with the Duckiematrix ](https://docs.duckietown.com/ente/duckietown-manual/50-duckiematrix/introduction-to-the-duckiematrix-virtual-environment.html)
##### Virtual Duckiebots: digital twins for Duckietown robots
Virtual Duckietown robots allow for a Duckietown robot’s full software stack to be run on a local machine in its own Docker environment, and allowing for the full simulation of any aspect of that Duckietown robot within the Duckiematrix, simplifying testing and improving portability to the real world Duckiebots.
[ Virtual Duckietown Robots ](https://docs.duckietown.com/ente/devmanual-duckiematrix/intermediate/virtual_duckietown_robots/intro.html)
##### Code refactoring for faster development
The code in the autonomy stack has been refactored so that the key algorithms are moved into libraries. This facilitates the creation of notebooks for experimentation and learning, as well as enabling the code to be more portable and disentangled from the ROS infrastructure, setting the stage for using other middleware (e.g., ROS2).
[  ](https://docs.duckietown.com/ente/duckietown-manual/70-developer-manual/code-hierarchy/duckietown-docker-code-hierarchy-explained.html)
[ Main images and repository ](https://docs.duckietown.com/ente/duckietown-manual/70-developer-manual/code-hierarchy/duckietown-docker-code-hierarchy-explained.html)
##### The Duckietown Manual: all information in a single place
All documentation and information have been consolidated in the Duckietown Manual, a single, **authoritative,** and **searchable** source.
The new Duckietown Manual is a great place to get started, as it contains step-by-step instructions on how to set up your computer,
assemble, calibrate, and operate a Duckiebot, along with troubleshooting tips. It moreover includes information for advanced users who wish to develop using Duckietown, pointers to code Documentation, as well as an instructor manual with pedagogical insights for teachers.
[ The Duckietown Manual ](https://docs.duckietown.com/ente/duckietown-manual/welcome-to-the-duckietown-manual.html)
[  ](https://docs.duckietown.com/ente/duckietown-manual/welcome-to-the-duckietown-manual.html)
##### Duckietown Postal Service (DTPS) and new development workflow
The Duckietown Postal Service (`DTPS`) is an HTTP/2 compatible message-passing system that bridges between the Duckietown robots and the environment, whether physical or digital. DTPS enables upgrading from ROS to ROS2, or the use of any other similar middleware, and makes Duckietown more compatible with all OSs.
In addition, a new development workflow has been implemented. The API for working with learning experiences (`dts code`) has been significantly improved over the previous version.
[ Duckietown Postal Service (DTPS) ](https://docs.duckietown.com/ente/opmanual-duckiebot/04-software-tools/duckietown-postal-service-dtps.html)
##### Duckiebot UI improvements
A few actuator and sensor interfaces were updated for improved usability and robot management, for example:
- [Image Viewer](https://docs.duckietown.com/ente/opmanual-duckiebot/02-operations/03-subsystem-testing/make-it-see-duckiebot-image-streaming.html): to better see what your Duckiebot sees;
- [Keyboard Controller](https://docs.duckietown.com/ente/opmanual-duckiebot/02-operations/03-subsystem-testing/make-it-move-duckiebot-keyboard-control.html): now including other sensor and actuator readings, in addition to odometry calibration inputs.
- [LED Controller](https://docs.duckietown.com/ente/opmanual-duckiebot/02-operations/03-subsystem-testing/make-it-shine-duckiebot-led-control.html): to intuitively control each LED’s color and intensity;
- [Intrinsic and Extrinsic calibrators](https://docs.duckietown.com/ente/opmanual-duckiebot/02-operations/04-calibrations/duckiebot-camera-calibration.html): to improve the camera calibration procedure, making it faster and more reproducible.
## Where are we going from here?
##### Coming soon: Self-Driving Cars with Duckietown 2025
A new edition of [Self-Driving Cars with Duckietown MOOC](https://duckietown.com/self-driving-cars-with-duckietown-mooc/), the world’s first robot autonomy massive open online course (MOOC) with hardware, will soon be announced. This new edition will be ente-based, support the Duckiematrix and be instructor-paced.
##### ROS 2 autonomy baseline and Python SDK interface
With DTPS enabling support for any middleware, translating the current ROS lane following pipeline into a ROS2 one is now a fun project. Coming out soon!
A Python SDK to interface Duckietown robots and the Duckiematrix is in the works as well.
##### Duckiematrix updates in development
**Duckiematrix map editor**
An app for creating and editing maps for the Duckiematrix.
**Duckiematrix Gym**
The integration of the Duckiematrix with [Gymnasium](https://gymnasium.farama.org/).
**Duckiedrone support for the Duckiematrix**
The addition of Virtual Duckiedrones and the integration of Duckiedrones with the Duckiematrix.
## How to get started with Duckietown?
While the legacy `daffy` version of Duckietown will stay up and be supported for the time being, it will not receive further updates. To upgrade your environment and your Duckiebots to the new ente version and start experiencing all the new features for free, see our guide [here](https://docs.duckietown.com/ente/duckietown-manual/10-setup/02-software/duckietown-shell-dts-installation.html#dt-account-switch-profile).
### About Duckietown
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** News
**Tags:** duckiematrix, ente
---
### [Sim2Real Lane Segmentation via Domain Adaptation](https://duckietown.com/sim2real-lane-segmentation-via-domain-adaptation/)
**Published:** July 31, 2025
**Author:** Duckietown Admin
**Excerpt:** Sim2Real lane segmentation using domain adaptation enables transfer from synthetic to real data with minimal labeled supervision in Duckietown.
**Content:**
##### General Information
- **Title**: Simulation to Real Domain Adaptation for Lane Segmentation
- **Authors**: Márton Tim, Márton Szemenyei, Róbert Moni
- **Institution**: Budapest University of Technology and Economics, Budapest, Hungary
- **Citation**: M. Tim, M. Szemenyei and R. Moni, "Simulation to Real Domain Adaptation for Lane Segmentation," 2020 23rd International Symposium on Measurement and Control in Robotics (ISMCR), Budapest, Hungary, 2020, pp. 1-6, doi: 10.1109/ISMCR51255.2020.9263406.
[ Paper ](https://ieeexplore.ieee.org/document/9263406)
[ Authors ](#authors)
[ Institution ](https://www.bme.hu/en)
# Sim2Real Lane Segmentation via Domain Adaptation
This embodied AI work investigates **Sim2Real transfer**: the process of applying ML agents trained in simulation to real-world environments, for **semantic lane segmentation** in mobile robotics using **domain adaptation techniques**.
![CycleGAN model used for domain adaptation from simulation to real images in Duckietown Sim2Real learning]()
CycleGAN-Based Domain Conversion from Simulation to Reality
![Qualitative comparison of domain adaptation models for Sim2Real lane detection in Duckietown]()
Visual Comparison of Domain Adaptation Models for Sim2Real Segmentation
![RGB histogram comparison before and after domain adaptation for Sim2Real transfer in Duckietown]()
Histogram Alignment for Domain Adaptation in Sim2Real Transfer
![Minimax Entropy method applied in Sim2Real domain adaptation for Duckietown lane segmentation]()
Semi-Supervised Domain Adaptation Using Minimax Entropy
![IoU comparison over training for Sim2Real domain adaptation methods on Duckietown segmentation]()
Validation IoU Trajectory of Sim2Real Domain Adaptation Methods
The study addresses the **distributional shift** between synthetic (simulated) and real-world data using **unsupervised** and **semi-supervised learning** approaches that minimize the need for manual annotation by learning from unlabeled data or limited labeled samples.
A **convolutional neural network (CNN)** with an **encoder-decoder architecture** is trained on labeled synthetic data generated in the [Duckietown Gym](https://github.com/duckietown/gym-duckietown "Duckietown Gym") and adapted to unlabeled real-world images captured in the physical Duckietown setup.
The method integrates:
- **Feature-level and pixel-level adaptation**, aligning internal representations and input appearance between domains to ensure consistent segmentation.
- **Adversarial training**, where a discriminator encourages the CNN to learn domain-invariant features.
- **Cycle-consistent generative adversarial networks (CycleGANs)**, which perform image-to-image translation to make synthetic images visually similar to real ones while preserving semantic structure.
- Evaluation using **mean Intersection over Union (mIoU)** and **pixel accuracy**, both standard metrics for assessing segmentation quality.
The results demonstrate that domain adaptation enables effective Sim2Real transfer for lane detection in Duckietown with minimal supervision advancing the deployment of robust, label-efficient perception systems in embedded robotics and autonomous navigation.
[ Learn about machine learning with Duckietown ](https://duckietown.com/educational-resources/)
## Highlights - Sim2Real lane segmentation via domain adaptation
Here is a visual tour of the implementation of lane segmentation via domain adaptation by the authors. For all the details, check out the [full paper](#links "Post resources").
[](https://duckietown.com/wp-content/uploads/2025/07/Histogram-Alignment-for-Domain-Adaptation-in-Sim2Real-Transfer.avif) Histogram Alignment for Domain Adaptation in Sim2Real Transfer [](https://duckietown.com/wp-content/uploads/2025/07/CycleGAN-Based-Domain-Conversion-from-Simulation-to-Reality.avif) CycleGAN-Based Domain Conversion from Simulation to Reality
[](https://duckietown.com/wp-content/uploads/2025/07/Semi-Supervised-Domain-Adaptation-Using-Minimax-Entropy.avif) Semi-Supervised Domain Adaptation Using Minimax Entropy [](https://duckietown.com/wp-content/uploads/2025/07/Visual-Comparison-of-Domain-Adaptation-Models-for-Sim2Real-Segmentation.avif) Visual Comparison of Domain Adaptation Models for Sim2Real Segmentation
[](https://duckietown.com/wp-content/uploads/2025/07/Validation-IoU-Trajectory-of-Sim2Real-Domain-Adaptation-Methods.avif) Validation IoU Trajectory of Sim2Real Domain Adaptation Methods
## Abstract
Here is the abstract of the work, directly in the words of the authors:
As the **cost of labelling and collecting real world data** remains an issue for companies, simulator training and transfer learning slowly evolved to be the foundation of many state-of the-art projects. In this paper these methods are applied in the Duckietown setup where self-driving agents can be developed and tested.
Our aim was to train a selected artificial neural network for right lane segmentation on simulator generated stream of images as a comparison baseline, then use domain adaptation to be more precise and stable in the real environment. We have tested and compared four knowledge transfer methods that included domain transformation using CycleGAN and semi-supervised domain adaptation via Minimax Entropy.
As the latter was previously untested in semantic segmentation according to our best knowledge, we have contributed to showing it is indeed possible and produces promising results. Finally we have shown that it could also create a model that fulfills our performance requirements of stability and accuracy.We show that the selected methods are equally eligible for the simulation to real transfer learning problem, and that the simplest method delivers the best performance.
## Conclusion - Sim2Real lane segmentation via domain adaptation
Here is the conclusion according to the authors of this paper:
Our goal was to create a stable and accurate right lane segmentation network by means of simulator data and domain adaptation techniques. We have tested and compared four knowledge transfer methods that included domain transformation using CycleGAN and semi-supervised domain adaptation via Minimax Entropy. We have shown that in the given scenario simulator-trained models have relatively good performance on real images, though their stability is a key weakness.
Our findings demonstrate that domain transformation using CycleGAN has limited applicability in segmentation tasks due to its distorting effect on road geometry, however the similarity between training and testing domains did result in increased stability.
Unfortunately, histogram matching failed in our case to improve on the baseline solution, producing similar results to CycleGAN.
We have observed that one of the simplest domain adaptation methods, source and target combined domain training helped to produce the best performing model according to numerical evaluation.
We implemented and demonstrated how semi-supervised domain adaptation via Minimax Entropy, a complex, entropybased adversarial method is applicable for segmentation tasks.
In the end, all the existing results were compared and evaluated with the conclusion that source and target combined domain training produced the best results of all investigated methods tied with SSDA via Minimax Entropy. Thereby, the usability of the latter method in segmentation tasks has also been proven.
##### Did this work spark your curiosity?
Check out the following works on vehicle autonomy with Duckietown:
- - [Adapting World Models with Latent-State Dynamics Residuals](https://duckietown.com/adapting-world-models-with-latent-state-dynamics-residuals/)
- [Semantic Image Segmentation Methods in the Duckietown Project](https://duckietown.com/vision-based-drl-autonomous-driving-agent-with-sim2real-transfer-li-okhrin/ "Vision-Based DRL Autonomous Driving Agent with Sim2Real Transfer")
- [Visual Monitoring of Swarms of Industrial Robots](https://duckietown.com/visual-monitoring-of-automated-guided-vehicles-in-duckietown/)
- [Deep Reinforcement and Transfer Learning for Robot Autonomy](https://duckietown.com/deep-reinforcement-and-transfer-learning-for-robot-autonomy/ "Deep Reinforcement and Transfer Learning for Robot Autonomy")
- [Enhancing Visual Domain Randomization with Real Images for Sim-to-Real Transfer](https://duckietown.com/enhancing-visual-domain-randomization-for-sim2real-transfer/ "Enhancing Visual Domain Randomization with Real Images for Sim-to-Real Transfer")
#### Project Authors
[  ](https://www.linkedin.com/in/marton-tim/?originalSubdomain=hu)
[Márton Tim](https://www.linkedin.com/in/marton-tim/?originalSubdomain=hu "Márton Tim") is currently working as a deep learning engineer at [Continental](https://www.continental.com/en/ "Continental"), Hungary.
[  ](https://www.linkedin.com/in/m%C3%A1rton-szemenyei-6959a3a9/)
[Márton Szemenyei](https://www.linkedin.com/in/m%C3%A1rton-szemenyei-6959a3a9/ "Márton Szemenyei") is an Associate Professor at [Budapest University of Technology and Economics](https://www.bme.hu/en "Budapest University of Technology and Economics"), Hungary.
[  ](https://ieeexplore.ieee.org/author/37332904400)
[Robert Moni](https://www.linkedin.com/in/robert-moni-942058100/ "Robert Moni") is currently working as a Senior Machine Learning Engineer at [Continental](https://www.continental.com/en/ "Continental"), Hungary.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** Research
**Tags:** CV
---
### [Robert Moni's experience after winning AI-DO 5](https://duckietown.com/robert-monis-experience-after-winning-ai-do-5/)
**Published:** January 16, 2021
**Author:** Liuxin
**Content:**
- [ Robert Moni ](https://www.linkedin.com/in/robert-moni-942058100/?originalSubdomain=hu)
- [ Budapest University of Technology and Economics ](https://www.bme.hu/?language=en)
- [ SmartLab AI ](https://smartlabai.medium.com/)
## An interview with Robert Moni
Robert is a Ph. D. student at the Budapest University of Technology and Economics.
His work focuses on deep learning and he has (co)authored [papers](https://scholar.google.com/citations?hl=it&user=gTc89ToAAAAJ) on reinforcement learning (RL), imitation learning (IL), and sim-to-real learning using for autonomous vehicles using Duckietown.
Robert and his team won the `LFV_multi` hardware [challenge](https://challenges.duckietown.com/v4/humans/challenges/aido5-LFV_multi-real-validation/leaderboard) of the 2020 AI Driving Olympics.
Today, Robert shares some of his thoughts with us!
### What brought you to work on AVs?
I started my journey in the world of AV’s in 2016 when I was hired at the automotive supplier company “Continental” in Romania. In 2018 I moved to Budapest, Hungary, to join Continental’s Deep Learning Competence Center where we develop novel perception methods for AVs.
In 2019, with the support of the company, I started my Ph.D. at Budapest University of Technology and Economics on the topic “Deep Reinforcement Learning in Complex environments”.
At this time, I crossed paths with the Duckietown environment. Continental bought 12 Duckiebots and supplementary materials to build our own Duckietown environment in a lab at the university.
### Tell us about you and your team
At the beginning of my Ph. D. and with the arrival of the Duckietown materials we established the “PIA” ([Professional Intelligence for Automotive](https://smartlabai.medium.com/pia-projects-perseverance-e3a659bfca82)) project with the aim to provide education and mentorship for undergrad and master students in the field on Machine Learning and AV.
In each semester since 2019 February I managed a team of 4-6 people developing their own solutions for AI-DO challenges. I wrote a short [blogpost](https://medium.com/@SmartLabAI/pia-project-achievements-at-aido5-a441a24484ef) presenting my team and our solutions submitted to AI-DO 5.
> "With the arrival of the Duckietown material we established the PIA project with the aim to provide education and mentorship for undergrad and master students in the field on Machine Learning and autonomous vehicles (AV)."
>
> Robert Moni
### What approach did you choose for AI-DO, and why?
I started to tackle the AI-DO challenges applying deep reinforcement learning (DRL) for driver policy learning and state representation learning (SRL) for sim2real transfer.
The reason for my chosen approach is my Ph. D. topic, and I plan to develop and test my hypotheses in the Duckietown environment.
### What are the hardest challenges that you faced in the competition?
In the beginning, there was a simple agent training task that caused some headaches: finding a working DRL method, composing a good reward function, preprocessing the observations to reduce the search space, and fine-tuning all the parameters. All these were challenges, but well-known ones in the field.
One unexpected challenge was the continuous updates of the gym-duckietown environment. While we are thrilled that the environment gets improved by the Duckietown team, we faced occasional breakdowns in our methods when applying them to the newest releases, which caused some frustration.
The biggest headache was caused by the different setups in the training and evaluation environments: in the evaluation environment, the images are dimmed while during training they are clear. Furthermore, the real world is full of nuisances – for example lags introduced by WiFi communication, which causes different outcomes in the real environment. This challenge can be mitigated to some degree with the algorithms running directly on the Duckiebot’s hardware, and by using a more powerful onboard computer, e.g., the Jetson Nano 2GB development board.
### Are you satisfied with the final outcome?
I am satisfied with the achievements of my team, which kept the resolve throughout the technical challenges faced.
I’m sure we would’ve done even better in the real-world challenge if we had seen our submission running earlier in the Autolab, so we could have adjusted our algorithms. We are going to work to bring one to our University in the next future.
### What are you going to change next time?
I believe the AI-DO competition as well as the Duckietown platform would improve through more powerful hardware. I hope to see Duckiebots (DB19) upgraded to support the new Jetson Nano hardware!
(Since the date of the interview, [Duckiebots model DB21](https://get.duckietown.com/products/duckiebot-db21?variant=40700056895663) supports Jetson Nano boards)
### Learn more about Duckietown
The Duckietown platform offers robotics and AI learning experiences.
Duckietown is modular, customizable and state-of-the-art. It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Find out more use cases ](https://www.duckietown.com/news/people-of-duckietown)
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** News, People
---
### [Interpretable Reinforcement Learning for Visual Policies](https://duckietown.com/interpretable-reinforcement-learning-for-visual-policies/)
**Published:** July 2, 2025
**Author:** Duckietown Admin
**Excerpt:** Explore interpretable reinforcement learning using self-supervised attention mechanisms in Duckietown to understand agent decision-making processes.
**Content:**
##### General Information
- **Title**: Self-Supervised Discovering of Interpretable Features for Reinforcement Learning
- **Authors**: Wenjie Shi, Gao Huang, Shiji Song, Zhuoyuan Wang, Tingyu Lin, Cheng Wu
- **Institution**: Tsinghua University, Beijing, China
- **Citation**: W. Shi, G. Huang, S. Song, Z. Wang, T. Lin and C. Wu, "Self-Supervised Discovering of Interpretable Features for Reinforcement Learning," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 5, pp. 2712-2724, 1 May 2022, doi: 10.1109/TPAMI.2020.3037898.
[ Paper ](https://ieeexplore.ieee.org/document/9259236)
[ Authors ](#authors)
[ Institution ](https://www.tsinghua.edu.cn/en/)
# Interpretable Reinforcement Learning for Visual Policies
![Diagram of a two-stage architecture for Interpretable Reinforcement Learning using Self-Supervised Attention Mechanisms in Duckietown]()
Figure 1. Two-Stage Framework for Interpretable Reinforcement Learning
![Heatmaps showing basic attention patterns for Interpretable Reinforcement Learning using Self-Supervised Attention Mechanisms in Duckietown RL agents]()
Figure 2. Visual Attention Patterns in Reinforcement Learning
![Graph comparing returns for Interpretable Reinforcement Learning with Self-Supervised Attention Mechanisms in Duckietown]()
Figure 3. Expert vs Mask Policy Return Comparison
![Comparison of Interpretable Reinforcement Learning with Self-Supervised Attention Mechanisms applied to Atari and Duckietown tasks]()
Figure 4. RL Performance on Atari Using Masked Inputs
![Visualization of masked attention in Interpretable Reinforcement Learning using Self-Supervised Attention Mechanisms on Duckietown maps]()
Figure 5. Attention Mask Visualizations on Duckietown Maps
![Saliency map comparison for Interpretable Reinforcement Learning with Self-Supervised Attention Mechanisms in Duckietown and Atari agents]()
Figure 6. Saliency Map Comparison Across Methods
![Mask evolution for Interpretable Reinforcement Learning using Self-Supervised Attention Mechanisms in Duckietown under increasing regularization]()
Figure 7. Effect of Regularization on Attention Mask
![Failure case showing attention drift in Interpretable Reinforcement Learning using Self-Supervised Attention Mechanisms on Duckietown corner turn]()
Figure 8. Attention Shift in Failure Scenario
![Example of Interpretable Reinforcement Learning mask evaluation using Self-Supervised Attention Mechanisms in Duckietown]()
Figure 9. Mask Evaluation Example on Duckietown
![Return vs attention quality graph for Interpretable Reinforcement Learning with Self-Supervised Attention Mechanisms in Duckietown]()
Figure 10. Relationship Between Attention Quality and Return
![3D plot showing PPO, SAC, and TD3 performance in Interpretable Reinforcement Learning using Self-Supervised Attention Mechanisms in Duckietown]()
Figure 11. Performance Comparison of PPO, SAC, and TD3 Agents
![Masked state sequences showing PPO, SAC, and TD3 agent behavior in Interpretable Reinforcement Learning with Self-Supervised Attention Mechanisms in Duckietown]()
Figure 12. Visual Comparison of PPO, SAC, and TD3 Agent Behavior
![3D chart comparing different actor architectures in Interpretable Reinforcement Learning with Self-Supervised Attention Mechanisms in Duckietown]()
Figure 13. Agent Performance by Actor Network Architecture
![Masked state sequence comparison of Unet, RefineNet, DeepLab-v3, and FC DenseNet for Interpretable Reinforcement Learning using Self-Supervised Attention Mechanisms in Duckietown]()
Figure 14. Masked State Visualization Across Actor Architectures
Reinforcement Learning (RL) has enabled solving complex problems, especially in relation to visual perception in robotics. An outstanding challenges is that of allowing humans to make sense of the decision making process, so to enable deployment in safety-critical applications such as, e.g., autonomous driving. This work focuses on the problem of interpretable reinforcement learning in vision-based agents.
In particular, this research introduces a self-supervised framework for interpretable reinforcement learning in vision-based agents. The focus lies in enhancing policy interpretability by generating precise attention maps through Self-Supervised Attention Mechanisms (SSAM).
The method does not rely on external labels and works using data generated by a pretrained RL agent. A self-supervised interpretable network (SSINet) is deployed to identify task-relevant visual features. The approach is evaluated across multiple environments, including Atari and Duckietown.
Key components of the method include:
- A two-stage training process using pretrained policies and frozen encoders
- Attention masks optimized using behavior resemblance and sparsity constraints
- Quantitative evaluation using FOR and BER metrics for attention quality
- Comparative analysis with gradient and perturbation-based saliency methods
- Application across various architectures and RL algorithms including PPO, SAC, and TD3
The proposed approach isolates relevant decision-making cues, offering insight into agent reasoning. In Duckietown, the framework demonstrates how visual interpretability can aid in diagnosing performance bottlenecks and agent failures, offering a scalable model for interpretable reinforcement learning in autonomous navigation systems.
[ Learn about machine learning with Duckietown ](https://duckietown.com/educational-resources/)
## Highlights - interpretable reinforcement learning for visual policies
Here is a visual tour of the implementation of interpretable reinforcement learning for visual policies by the authors. For all the details, check out the [full paper](#links "Post resources").
[](https://duckietown.com/wp-content/uploads/2025/07/Figure-1.-Two-Stage-Framework-for-Interpretable-Reinforcement-Learning-1.avif) Figure 1. Two-Stage Framework for Interpretable Reinforcement Learning [](https://duckietown.com/wp-content/uploads/2025/07/Figure-2.-Visual-Attention-Patterns-in-Reinforcement-Learning.avif) Figure 2. Visual Attention Patterns in Reinforcement Learning
[](https://duckietown.com/wp-content/uploads/2025/07/Figure-3.-Expert-vs-Mask-Policy-Return-Comparison-png.avif) Figure 3. Expert vs Mask Policy Return Comparison [](https://duckietown.com/wp-content/uploads/2025/07/Figure-4.-RL-Performance-on-Atari-Using-Masked-Inputs-png.avif) Figure 4. RL Performance on Atari Using Masked Inputs
[](https://duckietown.com/wp-content/uploads/2025/07/Figure-5.-Attention-Mask-Visualizations-on-Duckietown-Maps.avif) Figure 5. Attention Mask Visualizations on Duckietown Maps [](https://duckietown.com/wp-content/uploads/2025/07/Figure-6.-Saliency-Map-Comparison-Across-Methods.avif) Figure 6. Saliency Map Comparison Across Methods
[](https://duckietown.com/wp-content/uploads/2025/07/Figure-7.-Effect-of-Regularization-on-Attention-Mask.avif) Figure 7. Effect of Regularization on Attention Mask [](https://duckietown.com/wp-content/uploads/2025/07/Figure-8.-Attention-Shift-in-Failure-Scenario.avif) Figure 8. Attention Shift in Failure Scenario
[](https://duckietown.com/wp-content/uploads/2025/07/Figure-9.-Mask-Evaluation-Example-on-Duckietown-png.avif) Figure 9. Mask Evaluation Example on Duckietown [](https://duckietown.com/wp-content/uploads/2025/07/Figure-10.-Relationship-Between-Attention-Quality-and-Return.avif) Figure 10. Relationship Between Attention Quality and Return
[](https://duckietown.com/wp-content/uploads/2025/07/Figure-11.-Performance-Comparison-of-PPO-SAC-and-TD3-Agents.avif) Figure 11. Performance Comparison of PPO, SAC, and TD3 Agents [](https://duckietown.com/wp-content/uploads/2025/07/Figure-12.-Visual-Comparison-of-PPO-SAC-and-TD3-Agent-Behavior.avif) Figure 12. Visual Comparison of PPO, SAC, and TD3 Agent Behavior
[](https://duckietown.com/wp-content/uploads/2025/07/Figure-13.-Agent-Performance-by-Actor-Network-Architecture.avif) Figure 13. Agent Performance by Actor Network Architecture [](https://duckietown.com/wp-content/uploads/2025/07/Figure-14.-Masked-State-Visualization-Across-Actor-Architectures.avif) Figure 14. Masked State Visualization Across Actor Architectures
## Abstract
Here is the abstract of the work, directly in the words of the authors:
Deep reinforcement learning (RL) has recently led to many breakthroughs on a range of complex control tasks. However, the agent’s decision-making process is generally not transparent. The lack of interpretability hinders the applicability of RL in safety-critical scenarios. While several methods have attempted to interpret vision-based RL, most come without detailed explanation for the agent’s behavior. In this paper, we propose a self-supervised interpretable framework, which can discover interpretable features to enable easy understanding of RL agents even for non-experts. Specifically, a self-supervised interpretable network (SSINet) is employed to produce fine-grained attention masks for highlighting task-relevant information, which constitutes most evidence for the agent’s decisions. We verify and evaluate our method on several Atari 2600 games as well as Duckietown, which is a challenging self-driving car simulator environment. The results show that our method renders empirical evidences about how the agent makes decisions and why the agent performs well or badly, especially when transferred to novel scenes. Overall, our method provides valuable insight into the internal decision-making process of vision-based RL. In addition, our method does not use any external labelled data, and thus demonstrates the possibility to learn high-quality mask through a self-supervised manner, which may shed light on new paradigms for label-free vision learning such as self-supervised segmentation and detection.
## Conclusion - interpretable reinforcement learning for visual policies
Here is the conclusion according to the authors of this paper:
In this paper, we addressed the growing demand for human-interpretable vision-based RL from a fresh perspective. To that end, we proposed a general self-supervised interpretable framework, which can discover interpretable features for easily understanding the agent’s decision-making process. Concretely, a self-supervised interpretable network (SSINet) was employed to produce high-resolution and sharp attention masks for highlighting task-relevant information, which constitutes most evidence for the agent’s decisions. Then, our method was applied to render empirical evidences about how the agent makes decisions and why the agent performs well or badly, especially when transferred to novel scenes. Overall, our work takes a significant step towards interpretable vision-based RL. Moreover, our method exhibits several appealing benefits. First, our interpretable framework is applicable to any RL model taking as input visual images. Second, our method does not use any external labelled data. Finally, we emphasize that our method demonstrates the possibility to learn high-quality mask through a self-supervised manner, which provides an exciting avenue for applying RL to self automatically labelling and label-free vision learning such as self-supervised segmentation and detection.
##### Did this work spark your curiosity?
Check out the following works on vehicle autonomy with Duckietown:
- - [Adapting World Models with Latent-State Dynamics Residuals](https://duckietown.com/adapting-world-models-with-latent-state-dynamics-residuals/)
- [Semantic Image Segmentation Methods in the Duckietown Project](https://duckietown.com/vision-based-drl-autonomous-driving-agent-with-sim2real-transfer-li-okhrin/ "Vision-Based DRL Autonomous Driving Agent with Sim2Real Transfer")
- [Visual Monitoring of Swarms of Industrial Robots](https://duckietown.com/visual-monitoring-of-automated-guided-vehicles-in-duckietown/)
#### Project Authors
[  ](https://ieeexplore.ieee.org/author/37086608992)
[Wenjie Shi](https://ieeexplore.ieee.org/author/37086608992 "Wenjie Shi") received the BS degree from the School of Hydropower and Information Engineering, [Huazhong University of Science and Technology](https://english.hust.edu.cn/ "Huazhong University of Science and Technology"), Wuhan, China, in 2016. He is currently working toward the Ph.D. degree in control science and engineering from the Department of Automation, Institute of Industrial Intelligence and Systems, [Tsinghua University](https://www.tsinghua.edu.cn/en/ "Tsinghua University"), Beijing, China.
[  ](https://ieeexplore.ieee.org/author/37274328900)
[Gao Huang](https://www.linkedin.com/in/gao-huang-07102a93/) ([Member, IEEE](https://ieeexplore.ieee.org/author/37274328900)) received the B.S. degree in automation from [Beihang University](https://ev.buaa.edu.cn/ "Beihang University"), Beijing, China, in 2009, and the Ph.D. degree in automation from [Tsinghua University](https://www.tsinghua.edu.cn/en/ "Tsinghua University"), Beijing, in 2015. He is currently an Associate Professor with the Department of Automation, Tsinghua University.
[  ](https://ieeexplore.ieee.org/author/37332904400)
[Shiji Song](https://ieeexplore.ieee.org/author/37332904400 "Shiji Song") (Senior Member, IEEE) received the Ph.D. degree in mathematics from the Department of Mathematics, [Harbin Institute of Technology](https://en.hit.edu.cn/ "Harbin Institute of Technology"), Harbin, China, in 1996. He is currently a Professor at the Department of Automation, [Tsinghua University](https://www.tsinghua.edu.cn/en/ "Tsinghua University"), Beijing, China.
[  ](https://ieeexplore.ieee.org/author/37089351909)
[Zhuoyuan Wang](https://www.linkedin.com/in/zhuoyuan-jacob-wang/ "Zhuoyuan Wang Linkedin") ([IEEE](https://ieeexplore.ieee.org/author/37089351909)) is currently a Ph. D. student at Carnegie Mellon University, and holds a B.S. degree in control science and engineering in the Department of Automation, [Tsinghua University](https://www.tsinghua.edu.cn/en/ "Tsinghua University"), Beijing, China.
[  ](https://ieeexplore.ieee.org/author/37066522500)
[Tingyu Lin](https://ieeexplore.ieee.org/author/37066522500 "Tingyu Lin") received the B.S. degree and the Ph.D. degree in control system from the School of Automation Science and Electrical Engineering at [Beihang University](https://ev.buaa.edu.cn/ "Beihang University") in 2007 and 2014, respectively. He is now a Member of [China Simulation Federation](https://dc-china-simulation.researchcommons.org/ "China Simulation Federation") (CSF).
[  ](https://ieeexplore.ieee.org/author/37276423900)
[Cheng Wu](https://ieeexplore.ieee.org/author/37276423900 "Cheng Wu") received the M.Sc. degree in electrical engineering from [Tsinghua University](https://www.tsinghua.edu.cn/en/ "Tsinghua University"), Beijing, China, in 1966. He is currently a Professor with the Department of Automation, [Tsinghua University](https://www.tsinghua.edu.cn/en/ "Tsinghua University").
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** Research
**Tags:** CV
---
### [Object Detection on Duckiebots Using YOLOv5 Models](https://duckietown.com/object-detection-on-duckietown-robots-using-yolov5-models/)
**Published:** September 14, 2024
**Author:** Duckietown Admin
**Excerpt:** This study analyzes object detection on Duckiebots using YOLOv5 architecture, optimizing models with various techniques to achieve 97.7% accuracy in mAP.
**Content:**
##### General Information
- Analysis of Object Detection Models on Duckietown Robot Based on YOLOv5 Architectures
- Toan-Khoa Nguyen, Lien T. Vu, Viet Q. Vu, TiShu-Hao Liang, en-Dat Hoang, Minh-Quang Tran.
- National Taiwan University of Science and Technology, Taiwan.
- Nguyen, T.K., Vu, L.T., Vu, V.Q., Hoang, T.D., Liang, S.H. and Tran, M.Q., 2021. Analysis of object detection models on duckietown robot based on yolov5 architectures. International Journal of iRobotics, 4(4), pp.17-22.
[ Paper ](https://iroboticsjournal.org/index.php/irobotics/article/view/110)
[ Institution ](https://www.ntust.edu.tw/?Lang=en)
[ Authors ](#authors)
# Object Detection on Duckiebots Using YOLOv5 Models
Obstacle detection is about having autonomous vehicles perceive their surroundings, identify objects, and determine if they might conflict with the accomplishment of the robot’s task, e.g., navigating to reach a goal position.
Amongst the many applications of AI, object detection from images is arguably the one that experienced the most performance enhancement compared to “traditional approaches” such as color or blob detection.
Images are, from the point of view of a machine, nothing but (several) “tables” of numbers, where each number represents the intensity of light, at that location, across a channel (e.g., R, G, B for colored images).
Giving meaning to a cluster of numbers is not as easy as, for a human, it would be to identify a potential obstacle on the path. Machine learning-driven approaches have quickly outperformed traditional computer vision approaches at this task, strong of the abundant and cheap data for training made available by datasets and general imagery on the internet.
Various approaches (networks) for object detection have rapidly succeded in outperforming each other, and YOLO models particularly for their balance of computational efficiency and detection accuracy.
Learn about robot autonomy, and the difference between traditional and machine learning approaches, from the links below!
[ Duckietown Educational Resources ](https://duckietown.com/educational-resources/#machine-learning)
[ Try Object Detection in Duckietown ](https://github.com/duckietown/duckietown-lx/tree/mooc2022/object-detection)
## Abstract
In the author’s words:
Object detection technology is an essential aspect of the development of autonomous vehicles. The crucial first step of any autonomous driving system is to understand the surrounding environment.
In this study, we present an analysis of object detection models on the Duckietown robot based on You Only Look Once version 5 (YOLOv5) architectures. YOLO model is commonly used for neural network training to enhance the performance of object detection models.
In a case study of Duckietown, the duckies and cones present hazardous obstacles that vehicles must not drive into. This study implements the popular autonomous vehicles learning platform, Duckietown’s data architecture and classification dataset, to analyze object detection models using different YOLOv5 architectures. Moreover, the performances of different optimizers are also evaluated and optimized for object detection.
The experiment results show that the pre-trained of large size of YOLOv5 model using the Stochastic Gradient Decent (SGD) performs the best accuracy, in which a mean average precision (mAP) reaches 97.78%. The testing results can provide objective modeling references for relevant object detection studies.
## Highlights - Object Detection on Duckiebots Using YOLOv5 Models
Here is a visual tour of the work of the authors. For more details, check out the [full paper](#links "Post resources").
![Yolov5 object detection pic]()
Figure 1. Duckiebot and Obstacles: Cones and Duckies.
![Diagram of YOLOv5 architecture showing backbone, neck, and head sections for object detection.]()
Figure 2. YOLOv5 Architecture: Backbone, Neck, and Head Components.
![Graphs showing training results of YOLOv5s, including coordinate loss, objectness loss, classification loss, precision, and recall.]()
Figure 3. Training Results of Pre-Trained YOLOv5s for Object Detection.
![Bar chart comparing the performance of different YOLOv5 architectures for object detection.]()
Figure 4. Performance Comparison of YOLOv5 Architectures for Object Detection.
## Conclusion - Object Detection on Duckiebots Using YOLOv5 Models
Here are the conclusions from the authors of this paper:
“This paper presents an analysis of object detection models on the Duckietown robot based on YOLOv5 architectures. The YOLOv5 model has been successfully used to recognize the duckies and cones on the Duckietown. Moreover, the performances of different YOLOv5 architectures are analyzed and compared.
The results indicate that using the pre-trained model of YOLOv5 architecture with the SGD optimizer can provide excellent accuracy for object detection. The higher accuracy can also be obtained even with the medium size of the YOLOv5 model that enables to accelerate the computation of the system.
Furthermore, once the object detection model is optimized, it is integrated into the ROS in the Duckietown robot. In future works, it is potential to investigate the YOLOv5 with Layer-wise Adaptive Moments Based (LAMB) optimizer instead of SGD, applying repeated augmentation with Binary Cross-Entropy (BCE), and using domain adaptation technique.”
#### Project Authors
[  ](https://www.linkedin.com/in/toankhoa/)
[Toan-Khoa Nguyen](https://www.linkedin.com/in/toankhoa/) is currently working as an AI engineer at [FPT Software AI Center](), Vietnam.
[  ](https://mem.phenikaa-uni.edu.vn/vi/post/gioi-thieu/danh-sach-can-bogiang-vien/ts-vu-thi-lien)
[Lien T. Vu](https://mem.phenikaa-uni.edu.vn/vi/post/gioi-thieu/danh-sach-can-bogiang-vien/ts-vu-thi-lien) is with the Faculty of Mechanical Engineering and Mechatronics, [Phenikaa University](https://mem.phenikaa-uni.edu.vn/vi), Vietnam.
[  ](https://ieeexplore.ieee.org/author/37089322218)
[Viet Q. Vu](https://ieeexplore.ieee.org/author/37089322218) is with the Faculty of International Training, [Thai Nguyen University of Technology](http://en.tnu.edu.vn/), Vietnam.
[  ](https://www.linkedin.com/in/tien-dat-hoang-62718663/)
[Tien-Dat Hoang](https://www.linkedin.com/in/tien-dat-hoang-62718663/) is with the Faculty of International Training, [Thai Nguyen University of Technology](http://en.tnu.edu.vn/), Vietnam.
[  ](https://ieeexplore.ieee.org/author/37088514119)
[Shu-Hao Liang](https://ieeexplore.ieee.org/author/37088514119) is with the Center for Cyber-Physical System Innovation, [National Taiwan University of Science and Technology](https://www.ntust.edu.tw/?Lang=en), Taiwan.
[  ](https://ieeexplore.ieee.org/author/37088802152)
[Minh-Quang Tran](https://ieeexplore.ieee.org/author/37088802152) is with the Industry 4.0 Implementation Center, Center for Cyber-Physical System Innovation, [National Taiwan University of Science and Technology](https://www.ntust.edu.tw/?Lang=en), Taiwan and also with the Department of Mechanical Engineering, [Thai Nguyen University of Technology](http://en.tnu.edu.vn/), Vietnam.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Learn about Duckietown ](/)
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** AI, research
---
### [AI Driving Olympics Ready for Submissions](https://duckietown.com/ai-driving-olympics-ready-for-submissions/)
**Published:** October 14, 2018
**Author:** Liam Paull
**Content:**
Submissions to the AI Driving Olympics are officially open for the Lane Following task evaluated in simulation.
To make a submission, follow the instructions in the [AIDO-book](http://docs.duckietown.com/DT18/AIDO/out/index.html).
For example, a good place to start is the [Quick Start Guide](http://docs.duckietown.com/DT18/AIDO/out/quickstart.html).
For discussion please check out [the AIDO forums](https://duckietown.com/research/ai-driving-olympics/ "https://duckietown.com/research/ai-driving-olympics/").
**Categories:** Blog, Events, News
**Tags:** AI Driving Olympics, world
---
### [AI-DO I Interactive Tutorials](https://duckietown.com/ai-do-i-interactive-tutorials/)
**Published:** October 30, 2018
**Author:** Liam Paull
**Content:**
The AI Driving Olympics, presented by the Duckietown Foundation with help from our [partners and sponsors](/?page_id=2327) is now in full swing. Check out the [leaderboard](https://challenges.duckietown.com/)!
We now have templates for [ROS](http://docs.duckietown.com/DT18/AIDO/out/ros_template.html), [PyTorch](http://docs.duckietown.com/DT18/AIDO/out/pytorch_template.html), and [TensorFlow](http://docs.duckietown.com/DT18/AIDO/out/tensorflow_template.html), as well as an [agnostic template](http://docs.duckietown.com/DT18/AIDO/out/challenge_aido1_lf1_template_random.html).
We also have baseline implementation using the [classical pipeline](http://docs.duckietown.com/DT18/AIDO/out/ros_baseline.html), imitation learning with data from both [simulation](http://docs.duckietown.com/DT18/AIDO/out/embodied_il_sim.html) and [real](http://docs.duckietown.com/DT18/AIDO/out/embodied_il_logs.html) Duckietown logs, and [reinforcement learning](http://docs.duckietown.com/DT18/AIDO/out/embodied_rl.html).
We are excited to announce that we will be hosting a series of interactive tutorials for competitors to get started. These tutorials will be streamed live from our [Facebook page](https://www.facebook.com/duckietown).
See [ here ](https://duckietown.com/ai-do-i-interactive-tutorials/) for the full tutorial schedule.
**Categories:** Blog, Events
**Tags:** AI Driving Olympics, world
---
### [Survey on Testbeds for Vehicle Autonomy & Robot Swarms](https://duckietown.com/survey-on-testbeds-for-vehicle-autonomy-robot-swarms/)
**Published:** September 2, 2024
**Author:** Duckietown Admin
**Excerpt:** This paper surveys 17 small-scale testbeds for Connected and Vehicle Autonomy and Robot Swarms, detailing their characteristics and classifications.
**Content:**
##### General Information
- A Survey on Small-Scale Testbeds for Connected and Automated Vehicles and Robot Swarms
- Armin Mokhtarian, Jianye Xu, Patrick Scheffe, Maximilian Kloock, Simon Schäfer, Heeseung Bang, Viet-Anh Le, Sangeet Ulhas, Johannes Betz, Sean Wilson, Spring Berman, Liam Paull, Amanda Prorok, Bassam Alrifaee
- RWTH Aachen University, Germany
- Mokhtarian, Armin & Scheffe, Patrick & Kloock, Maximilian & Schäfer, Simon & Bang, Heeseung & Le, Viet-Anh & Sankaramangalam Ulhas, Sangeet & Betz, Johannes & Wilson, Sean & Berman, Spring & Prorok, Amanda & Alrifaee, Bassam. (2024). A Survey on Small-Scale Testbeds for Connected and Automated Vehicles and Robot Swarms. 10.13140/RG.2.2.16176.74248/1.
[ Paper ](https://www.researchgate.net/publication/378394694_A_Survey_on_Small-Scale_Testbeds_for_Connected_and_Automated_Vehicles_and_Robot_Swarms)
[ Institution ](https://www.rwth-aachen.de/)
[ Authors ](#authors)
[ CPM Website ](https://cpm.lrt.unibw.de/survey/)
# Survey on Testbeds for Vehicle Autonomy & Robot Swarms
“A Survey on Small-Scale Testbeds for Connected and Automated Vehicles and Robot Swarms“ by Armin Mokhtarian et al. offers a comparison of current small-scale testbeds for Connected and Automated Vehicles (CAVs), Vehicle Autonomy and Robot Swarms (RS).
As mentioned in , small-scale autonomous vehicle testbeds are paving the way to faster and more meaningful research and development in vehicle autonomy, embodied AI, and AI robotics as a whole.
Although small-scale, often made of off-the-shelf components and relatively low-cost, these platforms provide the opportunity for deep insights into specific scientific and technological challenges of autonomy.
Duckietown, in particular, is highlighted for its modular, miniature-scale smart-city environment, which facilitates the study of autonomous vehicle localization and traffic management through onboard sensors.
Learn about robot autonomy, traditional robotics autonomy architectures, agent training, sim2real, navigation, and other topics with Duckietown, starting from the link below!
[ Duckietown Educational Resources ](https://duckietown.com/educational-resources)
## Abstract
Connected and Automated Vehicles (CAVs) and Robot Swarms (RS) have the potential to transform the transportation and manufacturing sectors into safer, more efficient, sustainable systems.
However, extensive testing and validation of their algorithms are required. Small-scale testbeds offer a cost-effective and controlled environment for testing algorithms, bridging the gap between full-scale experiments and simulations. This paper provides a structured overview of characteristics of testbeds based on the sense-plan-act paradigm, enabling the classification of existing testbeds.
Its aim is to present a comprehensive survey of various testbeds and their capabilities. We investigated 17 testbeds and present our results on the public webpage [https://cpm.lrt.unibw.de/survey/](https://cpm.lrt.unibw.de/survey/ "cpm.lrt.unibw.de/survey/").
Furthermore, this paper examines seven testbeds in detail to demonstrate how the identified characteristics can be used for classification purposes.
## Highlights - Survey on Testbeds for Vehicle Autonomy & Robot Swarms
Here is a visual tour of the authors’ work. For more details, check out the [full paper](#links "Post resources") or the corresponding up-to-date [project website](https://cpm-remote.embedded.rwth-aachen.de/testbeds "A Survey on Small-Scale Testbeds for Connected and Automated Vehicles and Robot Swarms,").
![Collage of various small-scale testbeds for Connected and Automated Vehicles (CAVs), Vehicle Autonomy and Robot Swarms, highlighting different setups and testing environments.]()
Figure 1. Collage showcasing diverse testbeds in the realm of Connected and Automated Vehicles and Robot Swarms.
![Screenshot of a webpage displaying a list of testbeds for Connected and Automated Vehicles and Robot Swarms, as investigated in a research study.]()
Figure 2. Screenshot of Public Webpage Listing Investigated Testbeds in Connected Vehicles & Robot Swarms Study.
![Cyber-Physical Mobility Lab at RWTH Aachen University, featuring testbeds for research on Connected and Automated Vehicles and Robot Swarms.]()
Figure 3. Cyber-Physical Mobility Lab at RWTH Aachen University.
![IDS Scaled Smart City testbed at Cornell University, designed for research in connected vehicles and smart city technologies.]()
Figure 4. IDS Scaled Smart City at Cornell University.
![Robotarium testbed at Georgia Institute of Technology, featuring multiple small robots in a collaborative swarm setup.]()
Figure 5. Robotarium Testbed at Georgia Institute of Technology.
![Cambridge Minicars testbed at the Prorok Lab, Cambridge University, showcasing miniature vehicles for multi-agent systems research.]()
Figure 6. Cambridge Minicars at the Prorok Lab at Cambridge University.
![Go-CHART testbed at Arizona State University, featuring scaled autonomous vehicles for testing control and coordination strategies.]()
Figure 7. The Go-CHART at Arizona State University.
![F1TENTH vehicle built at the University of Pennsylvania, a scaled-down autonomous race car used for research and competitions.]()
Figure 8. An exemplar F1TENTH vehicle built at the University of Pennsylvania.
![Duckietown testbed at MIT, featuring miniature autonomous vehicles navigating a small-scale city environment with roads and traffic signs.]()
Figure 9. Duckietown at Massachusetts Institute of Technology.
## Conclusion - Survey on Testbeds for Vehicle Autonomy & Robot Swarms
Here are the conclusions from the authors of this paper:
“This survey provides a detailed overview of small-scale CAV/RS testbeds, with the aim of helping researchers in these fields to select or build the most suitable testbed for their experiments and to identify potential research focus areas. We structured the survey according to characteristics derived from potential use cases and research topics within the sense-plan-act paradigm.
Through an extensive investigation of 17 testbeds, we have evaluated 56 characteristics and have made the results of this analysis available on our [webpage](https://cpm-remote.embedded.rwth-aachen.de/testbeds). We invited the testbed creators to assist in the initial process of gathering information and updating the content of this webpage. This collaborative approach ensures that the survey maintains its relevance and remains up to date with the latest developments.
The ongoing maintenance will allow researchers to access the most recent information. In addition, this paper can serve as a guide for those interested in creating a new testbed. The characteristics and overview of the testbeds presented in this survey can help identify potential gaps and areas for improvement.
One ongoing challenge that we identified with small-scale testbeds is the enhancement of their ability to accurately map to realworld conditions, ensuring that experiments conducted are as realistic and applicable as possible.
Overall, this paper provides a resource for researchers and developers in the fields of connected and automated vehicles and robot swarms, enabling them to make informed decisions when selecting or replicating a testbed and supporting the advancement of testbed technologies by identifying research gaps.”
#### Project Authors
[  ](https://www.linkedin.com/in/armin-mokhtarian-24b419240/)
[Armin Mokhtarian](https://www.linkedin.com/in/armin-mokhtarian-24b419240/) is currently working as a Research Associate & PhD Candidate at [RWTH Aachen University](https://www.rwth-aachen.de/), Germany.
[  ](https://www.linkedin.com/in/patrick-scheffe)
[Patrick Scheffe](https://www.linkedin.com/in/patrick-scheffe) is a Research Associate at [Lehrstuhl Informatik 11 – Embedded Software](https://embedded.rwth-aachen.de/), Germany.
[  ](https://www.linkedin.com/in/dr-maximilian-kloock-a1474b188/)
[Maximilian Kloock](https://www.linkedin.com/in/dr-maximilian-kloock-a1474b188/) is working as a Team Manager Advanced Battery Management System Technologies at [FEV Europe](https://www.fev.com/en), Germany.
[  ](https://www.linkedin.com/in/simon-schaefer-abc123/?originalSubdomain=de)
[Simon Schäfer](https://www.linkedin.com/in/simon-schaefer-abc123/?originalSubdomain=de) is a Visiting Researcher at [Faculty of Engineering, University of Alberta](https://www.uab.ca/engineering), Canada.
[  ](https://www.linkedin.com/in/heeseung-bang/?locale=en_US)
[Heeseung Bang](https://www.linkedin.com/in/heeseung-bang/?locale=en_US) is currently a Postdoctoral Associate at [Cornell University](http://www.cornell.edu/), USA.
[  ](https://www.linkedin.com/in/viet-anh-le-22831718b/)
[Viet-Anh Le](https://www.linkedin.com/in/viet-anh-le-22831718b/) is a Visiting Graduate Student at [Cornell University](http://www.cornell.edu/), USA.
[  ](https://www.linkedin.com/in/sulhas/)
[Sangeet Ulhas](https://www.linkedin.com/in/sulhas/) is a PhD candidate at [Ira A. Fulton Schools of Engineering at Arizona State University](http://engineering.asu.edu/), USA.
[  ](https://www.linkedin.com/in/johannes-betz-254049107/)
[Johannes Betz](https://www.linkedin.com/in/johannes-betz-254049107/) is a Assistant Professor at [Technische Universität München](https://www.tum.de/), Germany.
[  ](https://ieeexplore.ieee.org/author/37087241901)
[Sean Wilson](https://ieeexplore.ieee.org/author/37087241901) is a Senior Research Engineer at [Georgia Institute of Technology](http://www.gtri.gatech.edu/), USA.
[  ](https://search.asu.edu/profile/1943720)
[Spring Berman](https://search.asu.edu/profile/1943720) is an Associate Professor at [Arizona State University](https://www.asu.edu/), USA.
[  ](https://www.linkedin.com/in/liam-paull-83a5442b/)
[Liam Paull](https://www.linkedin.com/in/liam-paull-83a5442b/) is an Associate Professor at [Université de Montréal](http://www.umontreal.ca/), Canada and he is also the Chief Education Officer at [Duckietown](https://duckietown.com/), USA.
[  ](https://www.linkedin.com/in/aprorok/?originalSubdomain=uk)
[Amanda Prorok](https://www.linkedin.com/in/aprorok/?originalSubdomain=uk) is an associate professor at [University of Cambridge](https://www.cam.ac.uk/), UK.
[  ](https://www.linkedin.com/in/bassam-alrifaee-5b592aa6/?originalSubdomain=de)
[Bassam Alrifaee](https://www.linkedin.com/in/bassam-alrifaee-5b592aa6/?originalSubdomain=de) is a Professor at [Bundeswehr University Munich](https://www.unibw.de/), Germany.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Learn about Duckietown ](http://www.duckietown.com)
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** AI, research
---
### [Duckiebots are ready to conquer the world!](https://duckietown.com/the-duckiebots-are-ready-to-conquer-the-world/)
**Published:** August 1, 2018
**Author:** Andrea Censi
**Content:**
### Dear friends of Duckietown:
We are excited to bring you tremendous news about the Duckietown project.
In the past years we have had the support from many enthusiastic individuals who have donated their time and efforts to help the Duckietown project grow, and grown it has!
Duckietown started at MIT in 2016 – almost two years ago. Now Duckietown classes have been taught in 10 countries with more than 700 alumni.
The last months have been a transformative period for the project, as we prepare to jump to the next level in terms of scope and reach.
#### The Duckietown Foundation
We have established the Duckietown Foundation, a non-profit entity that will lead the Duckietown project.
Our [mission](https://www.duckietown.com/about/mission): make the world excited about the beauty, the fun, the importance, and the challenges of robotics and artificial intelligence, through learning experiences that are tangible, accessible, and inclusive.
The Duckietown Foundation will serve as the coordination point for the development of Duckietown. As a non-profit, the foundation can accept donations from individuals and companies for the promotion of affordable and fun robotics learning programs around the world.
#### A Kickstarter
We are organizing a Kickstarter to make it easier for people to obtain Duckiebots and Duckietowns.
This solves the biggest hurdle so on reproducing the Duckietown experience: the the lack of a one-click solution to acquire the hardware.
Also, working with thousands of pieces allows to drive down the price and to design our own custom boards.
See: [Our Kickstarter](https://www.kickstarter.com/projects/163162211/duckietown-a-playful-road-to-learning-robotics-and)
#### A donation program
As much as we aim to have affordable hardware, in certain parts of the world the only realistic price is $0.
That is why we have included a donate-a-Duckiebot and donate-a-class program through the Kickstarter.
Become a friend of Duckietown and support the distribution of low-cost and playful AI and robotics education to even more schools across the globe by backing our Kickstarter campaign.
To learn more about how to support Duckietown, reach out to info@duckietown.com.
#### A new website…
We’ve designed a new website that better serves users of the platform by offering support forums and more organized access to the teaching materials.
See: The new forums.
See: New “duckumentation” site [docs.duckietown.com](https://docs.duckietown.com/)
#### … and 700 more new websites
We want people to share their Duckietown experiences with other Duckie-enthusiasts, whether they be far or near. That’s now possible through upwards of 700 “community” subsites, each with a blog and a forum.
For more information, see the post Communities sites launched.
#### The AI Driving Olympics
In addition to its role as an education platform, Duckietown is a useful research tool.
We are happy to announce that Duckietown is the official platform for the AI Driving Olympics, a machine learning competition to be held at NIPS 2018 and ICRA 2019, the two largest machine learning and robotics conferences in the world. We challenge you to put your coding to the test and join the competition.
### That’s all for now! Thanks for listening –
The Duckietown project relies on an active and engaged community, which is why we want you to stay involved! Support robotics education and research – Sign up on our website! Back our kickstarter! Compete in the AI Driving Olympics!
For any additional information of if you would like to help us in other ways, please [see here for how to help us](https://www.duckietown.com).
**Categories:** Blog
**Tags:** world
---
### [Kicking off the Duckietown Donation program with Cali, Colombia](https://duckietown.com/kicking-off-the-duckietown-donation-program/)
**Published:** August 22, 2018
**Author:** Andrea Censi
**Content:**
 Our first donation of a class kit goes to Cali, Colombia.
We’ve reached our [Kickstarter](https://www.kickstarter.com/projects/163162211/duckietown-a-playful-road-to-learning-robotics-and?ref=cytuw0) goal!
This is great news because it means that we can kick off our donation program, with our first donation of a Class Kit, to students at the Universidad Autónoma de Occidente in [Cali, Colombia.](https://en.wikipedia.org/wiki/Cali)
### Why a donation program?
Artificial Intelligence and Robotics are the sciences of the future, which is why we want everyone to have the chance to play and learn with Duckietown. While we design our robot platform to be as inexpensive as possible, we realize that cost might be an obstacle for educators or students with limited resources.
That is why we have designed a donation program where individuals, organizations or companies can make Duckietown truly accessible to all. Everybody can support STEM education by donating Duckiebots, or an entire Class Kit, to deserving individuals or educators.
## ### Our first recipient
Our first recipient is **Prof. Victor Romero Cano**, a professor from the [Universidad Autónoma de Occidente](http://www.uao.edu.co/) in Cali, Colombia.
Victor has a Ph.D. in field robotics obtained at the University of Sydney, Australia. He teaches two courses at his institution, and supervises over 40 undergraduate students who are working towards their final research projects.
Victor will teach two classes using the Duckietown platform. The first is an introductory class to robotics, covering kinematic analysis, teleoperation, control and autonomous navigation for wheeled robots. The second class is more specifically about robotic perception, and will go in detail about mapping and SLAM (simultaneous localization and mapping), covering lane detection as well as object detection, recognition and tracking.
Victor’s first Duckietown class starts in January 2019. We welcome him to the community and look forward to hearing about his journey!
You can help us sponsor more donations by sponsoring[ our Kickstarter](https://www.kickstarter.com/projects/163162211/duckietown-a-playful-road-to-learning-robotics-and?ref=cytuw0).
**Categories:** dep-News, outreach
**Tags:** cali, class kit, colombia, donation, education, news, outreach, undergraduate, world
---
### [Reproducible Sim-to-Real Traffic Signal Control Environment](https://duckietown.com/reproducible-sim-to-real-traffic-signal-control-environment/)
**Published:** June 12, 2025
**Author:** Duckietown Admin
**Excerpt:** A low-cost sim-to-real testbed in Duckietown for evaluating traffic signal control policies with reproducible experiments and real-world sensors.
**Content:**
##### General Information
- **Title**: Reproducible and Low-cost Sim-to-Real Environment for Traffic Signal Control
- **Authors**: Yiran Zhang, Khoa Vo, Longchao Da, Tiejin Chen, Xiaoou Liu, Hua Wei
- **Institution**: Arizona State University, USA
- **Citation**: Zhang, Y., Vo, K., Da, L., Chen, T., Liu, X. and Wei, H., 2025, May. Reproducible and Low-cost Sim-to-Real Environment for Traffic Signal Control. In Proceedings of the ACM/IEEE 16th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2025) (pp. 1-2).
[ Paper ](https://dl.acm.org/doi/10.1145/3716550.3725161)
[ Authors ](#authors)
[ Institution ](https://www.asu.edu/)
# Reproducible Sim-to-Real Traffic Signal Control Environment
As urban environments become increasingly populated and automobile traffic soars, with US citizens spending on average 54 hours a year stuck on the roads, active traffic control management promises to mitigate traffic jams while maintaining (or improving) safety.
[LibSignal++](https://arxiv.org/abs/2211.10649 "LibSignal++ Duckietown-based testbed") is a Duckietown-based testbed for reproducible and low-cost sim-to-real evaluation of traffic signal control (TSC) algorithms. Using Duckietown enables consistent, small-scale deployment of both rule-based and learning-based TSC models.
LibSignal++ integrates visual control through camera-based sensing and object detection via the [YOLO-v5](https://github.com/ultralytics/yolov5 "YOLO-v5") model. It features modular components, including Duckiebots, signal controllers, and an indoor positioning system for accurate vehicle trajectory tracking. The testbed supports dynamic scenario replication by enabling both manual and automated manipulation of sensor inputs and road layouts.
Key aspects of the research include:
- Sim-to-real pipeline for Reinforcement Learning (RL)-based traffic signal control training and deployment
- Multi-simulator training support with [SUMO](https://github.com/LucasAlegre/sumo-rl "SUMO"), [CityFlow](https://cityflow-project.github.io/ "CityFlow"), and [CARLA](https://carla.readthedocs.io/en/0.9.13/tuto_G_rllib_integration/ "CARLA")
- Reproducibility through standardized and controllable physical components
- Integration of real-world sensors and visual control systems
- Comparative evaluation using rule-based policies on 3-way and 4-way intersections
The work concludes with plans to extend to Machine Learning (ML)-based TSC models and further sim-to-real adaptation.
[ Learn about reinforcement learning with Duckietown ](https://duckietown.com/educational-resources/#machine-learning)
## Highlights - Reproducible Sim-to-Real Traffic Signal Control Environment
Here is a visual tour of the sim-to-real work of the authors. For all the details, check out the [full paper](#links "Post resources").
![Duckietown sim-to-real testbed camera views of traffic intersections]()
Real-Time Camera Views of Intersections in LibSignal++
![Sim-to-real Duckietown-based traffic control testbed with Duckiebot and signal setup]()
Physical demonstration of LibSignal++ testbed
![Sim-to-real RL training pipeline based on Duckietown testbed for traffic signal control]()
Pipeline to train RL agents in simulators, sim-to-real transfer, and real testbed deployment and test
[](https://duckietown.com/wp-content/uploads/2025/06/Physical-demonstration-of-LibSignal-testbed.avif) Physical demonstration of LibSignal++ testbed
[](https://duckietown.com/wp-content/uploads/2025/06/Pipeline-to-train-RL-agents-in-simulators-sim-to-real-transfer-and-real-testbed-deployment-and-test.avif) Pipeline to train RL agents in simulators, sim-to-real transfer, and real testbed deployment and test
[](https://duckietown.com/wp-content/uploads/2025/06/The-camera-views-of-two-types-of-intersections.avif) Real-Time Camera Views of Intersections in LibSignal++
## Abstract
Here is the abstract of the work, directly in the words of the authors:
This paper presents a unique sim-to-real assessment environment for traffic signal control (TSC), LibSignal++, featuring a 14-ft by 14-ft scaled-down physical replica of a real-world urban roadway equipped with realistic traffic sensors such as cameras, and actual traffic signal controllers. Besides, it is supported by a precise indoor positioning system to track the actual trajectories of vehicles. To generate various plausible physical conditions that are difficult to replicate with computer simulations, this system supports automatic sensor manipulation to mimic observation changes and also supports manual adjustment of physical traffic network settings to reflect the influence of dynamic changes on vehicle behaviors. This system will enable the assessment of traffic policies that are otherwise extremely difficult to simulate or infeasible for full-scale physical tests, providing a reproducible and low-cost environment for sim-to-real transfer research on traffic signal control problems.
## Results
Three traffic control policies were tested over a number of experiment repetitions, evaluating each time traffic throughput, average vehicle waiting times, and vehicle battery consumption. Standard deviations for all policies were found to be within acceptable ranges, leading the authors to confirm the ability of the testbed to deliver reproducible results within controlled environments.
##### Did this work spark your curiosity?
Check out the follow works on machine learning with Duckietown:
- - [Sim2Real Transfer of Multi-Agent Policies for Self-Driving](https://duckietown.com/sim2real-transfer-of-multi-agent-policies-for-self-driving/ "Sim2Real Transfer of Multi-Agent Policies for Self-Driving")
- [Vision-Based DRL Autonomous Driving Agent with Sim2Real Transfer](https://duckietown.com/vision-based-drl-autonomous-driving-agent-with-sim2real-transfer-li-okhrin/ "Vision-Based DRL Autonomous Driving Agent with Sim2Real Transfer")
- [Application of PID controller and CNN to control Duckiebot robot](https://duckietown.com/pid-and-convolutional-neural-network-cnn-in-duckietown/ "Application of PID controller and CNN to control Duckiebot robot")
#### Project Authors
Yiran Zhang is associated with the [Arizona State University](https://www.asu.edu/ "Arizona State University"), USA.
Khoa Vo is associated with the [Arizona State University](https://www.asu.edu/ "Arizona State University"), USA.
[  ](https://search.asu.edu/profile/4886790)
[Longchao Da](https://search.asu.edu/profile/4886790 "Longchao Da") is pursuing his Ph.D. at the [Arizona State University](https://www.asu.edu/ "Arizona State University"), USA.
[  ](https://tiejin98.github.io/)
[Tiejin Chen](https://tiejin98.github.io/ "Tiejin Chen") is pursuing his Ph.D. at the [Arizona State University](https://www.asu.edu/ "Arizona State University"), USA.
[  ](https://xiao0o0o.github.io/)
[Xiaoou Liu](https://xiao0o0o.github.io/ "Xiaoou Liu") is pursuing her Ph.D. at the [Arizona State University](https://www.asu.edu/ "Arizona State University"), USA.
[  ](https://search.asu.edu/profile/3095662)
[Hua Wei](https://search.asu.edu/profile/3095662 "Hua Wei") is an Assistant Professor at the School of Computing and Augmented Intelligence, [Arizona State University](https://www.asu.edu/ "Arizona State University"), USA.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** Research
**Tags:** CV, DL
---
### [Visual Feedback for Autonomous Lane Tracking in Duckietown](https://duckietown.com/visual-feedback-for-lane-tracking-in-duckietown/)
**Published:** June 20, 2025
**Author:** Duckietown Admin
**Excerpt:** Duckietown vehicle autonomy: how to get an off-the-shelf self-driving car to navigate autonomously a model urban environment using only visual feedback.
**Content:**
##### General Information
- **Title**: Lane Following with a Duckiebot Vehicle using Visual Feedback
- **Authors**: Oscar Castro, Axel Céspedes, Roosevelt Ubaldo, Oscar E. Ramos
- **Institution**: Universidad de Ingenieria y Tecnologia - UTEC Lima, Peru
- **Citation**: O. Castro, A. Céspedes, R. Ubaldo and O. E. Ramos, "Lane Following with a Duckiebot Vehicle using Visual Feedback," 2019 IEEE Sciences and Humanities International Research Conference (SHIRCON), Lima, Peru, 2019, pp. 1-4, doi: 10.1109/SHIRCON48091.2019.9024875.
[ Paper ](https://ieeexplore.ieee.org/document/9024875)
[ Authors ](#authors)
[ Institution ](https://utec.edu.pe/)
# Visual Feedback for Autonomous Lane Tracking in Duckietown
How can vehicle autonomy be achieved by relying only on visual feedback from the onboard camera?
This work presents an implementation of lane following for the Duckietbot (DB17) using visual feedback as the only onboard sensor. The approach relies on real-time lane detection, and pose estimation, eliminating the need for wheel encoders.
The onboard computation is provided by a Raspberry Pi, which performs low-level motor control, while high-level image processing and decision-making are offloaded to an external ROS-enabled computer.
The key technical aspects of the implemented autonomy pipeline include:
- Camera calibration to correct fisheye lens distortion;
- HSV-based image segmentation for lane line detection;
- Aerial perspective transformation for geometric consistency;
- Histogram-based color separation of continuous and dashed lines;
- Piecewise polynomial fitting for path curvature estimation;
- Closed-loop motion control based on computed linear and angular velocities.
The methodology demonstrates the feasibility of using camera-based perception to control robot motion in structured environments. By using Duckiebot and Duckietown as the development platform, this work is another example of how to bridge the gap between real-world testing and cost-effective prototyping, making vehicle autonomy research more accessible in educational and research contexts.
[ Learn about machine learning with Duckietown ](https://duckietown.com/educational-resources/)
## Highlights - visual feedback for lane tracking in Duckietown
Here is a visual tour of the implementation of vehicle autonomy by the authors. For all the details, check out the [full paper](#links "Post resources").
![Real-time visual feedback overlay for vehicle autonomy in Duckietown with Duckiebot]()
Real-Time Lane Overlay
![Visual feedback communication structure for vehicle autonomy in Duckietown using Duckiebot]()
ROS Node Communication
![Visual feedback control system enabling vehicle autonomy in Duckietown]()
System Workflow Overview
![Duckiebot camera visual feedback preprocessing for vehicle autonomy in Duckietown]()
Image Distortion Correction
![Histogram analysis for lane segmentation using visual feedback in Duckiebot for vehicle autonomy in Duckietown]()
Lane Histogram Segmentation
![Aerial view transformation using visual feedback for vehicle autonomy in Duckietown with Duckiebot]()
Perspective Warping Process
![Visual feedback processing in Duckiebot for vehicle autonomy in Duckietown using HSV filtering and edge detection]()
HSV and Edge Detection
![Polynomial segmentation using visual feedback in Duckiebot for vehicle autonomy in Duckietown]()
Polynomial Adjustment Techniques
![Duckiebot demonstrating visual feedback for vehicle autonomy in Duckietown]()
Duckiebot used at UTEC
[](https://duckietown.com/wp-content/uploads/2025/06/Real-Time-Lane-Overlay.avif) Real-Time Lane Overlay [](https://duckietown.com/wp-content/uploads/2025/06/ROS-Node-Communication-png.avif) ROS Node Communication
[](https://duckietown.com/wp-content/uploads/2025/06/System-Workflow-Overview.avif) System Workflow Overview [](https://duckietown.com/wp-content/uploads/2025/06/Image-Distortion-Correction.avif) Image Distortion Correction
[](https://duckietown.com/wp-content/uploads/2025/06/Lane-Histogram-Segmentation-png.avif) Lane Histogram Segmentation [](https://duckietown.com/wp-content/uploads/2025/06/Perspective-Warping-Process.avif) Perspective Warping Process
[](https://duckietown.com/wp-content/uploads/2025/06/HSV-and-Edge-Detection-png.avif) HSV and Edge Detection [](https://duckietown.com/wp-content/uploads/2025/06/Polynomial-Adjustment-Techniques-png.avif) Polynomial Adjustment Techniques
[](https://duckietown.com/wp-content/uploads/2025/06/Duckiebot-and-Duckietown-circuit-at-UTEC.avif) Duckiebot used at UTEC
## Abstract
Here is the abstract of the work, directly in the words of the authors:
The autonomy of a vehicle can be achieved by a proper use of the information acquired with the sensors. Real-sized autonomous vehicles are expensive to acquire and to test on; however, the main algorithms that are used in those cases are similar to the ones that can be used for smaller prototypes. Due to these budget constraints, this work uses the Duckiebot as a testbed to try different algorithms as a first step to achieve full autonomy. This paper presents a methodology to properly use visual feedback, with the information of the robot camera, in order to detect the lane of a circuit and to drive the robot accordingly.
## Conclusion - visual feedback for lane tracking in Duckietown
Here is the conclusion according to the authors of this paper:
Autonomous cars are currently a vast research area. Due to this increase in the interest of these vehicles, having a costeffective way to implement algorithms, new applications, and to test them in a controlled environment will further help to develop this technology. In this sense, this paper has presented a methodology for following a lane using a cost-effective robot, called the Duckiebot, using visual feedback as a guide for the motion. Although the whole system was capable of detecting the lane that needs to be followed, it is still sensitive to illumination conditions. Therefore, in places with a lot of lighting and brightness variations, the lane recognition algorithm can affect the autonomy of the vehicle.
As future work, machine learning, and particularly convolutional neural networks, is devised as a means to develop robust lane detectors that are not sensitive to brightness variation. Moreover, more than one Duckiebot is intended to drive simultaneously in the Duckietown.
##### Did this work spark your curiosity?
Check out the following works on vehicle autonomy with Duckietown:
- - [Adapting World Models with Latent-State Dynamics Residuals](https://duckietown.com/adapting-world-models-with-latent-state-dynamics-residuals/)
- [Semantic Image Segmentation Methods in the Duckietown Project](https://duckietown.com/vision-based-drl-autonomous-driving-agent-with-sim2real-transfer-li-okhrin/ "Vision-Based DRL Autonomous Driving Agent with Sim2Real Transfer")
- [Visual Monitoring of Swarms of Industrial Robots](https://duckietown.com/visual-monitoring-of-automated-guided-vehicles-in-duckietown/)
#### Project Authors
[  ](https://www.linkedin.com/in/oscar-castro-ml/?locale=es_ES)
[Oscar Castro](https://www.linkedin.com/in/oscar-castro-ml/?locale=es_ES "Oscar Castro") is currently working at [Blume](https://www.blume.pe/ "Blume"), Peru.
[  ](https://www.linkedin.com/in/axel-cespedes-duran/?originalSubdomain=pe)
[Axel Eliam Céspedes Duran](https://www.linkedin.com/in/axel-cespedes-duran/?originalSubdomain=pe "Axel Eliam Céspedes Duran") is currently working as a Laboratory Professor of the Industrial Instrumentation course at the [UTEC – Universidad de Ingeniería y Tecnología](https://utec.edu.pe/ "UTEC - Universidad de Ingeniería y Tecnología"), Peru.
[  ](https://scholar.google.com/citations?user=fMeyG44AAAAJ&hl=es)
[Roosevelt Jhans Ubaldo Chavez](https://scholar.google.com/citations?user=fMeyG44AAAAJ&hl=es "Roosevelt Jhans Ubaldo Chavez") is currently working as a Laboratory Professor of the Industrial Instrumentation course at the [UTEC – Universidad de Ingeniería y Tecnología](https://utec.edu.pe/ "UTEC - Universidad de Ingeniería y Tecnología"), Peru.
[  ](https://ieeexplore.ieee.org/author/38238840400)
[Oscar E. Ramos](https://ieeexplore.ieee.org/author/38238840400 "Oscar E. Ramos") is currently working toward the Ph.D. degree in robotics with the Laboratory for Analysis and Architecture of Systems, Centre National de la Recherche Scientifique, [University of Toulouse](https://en.univ-toulouse.fr/ "University of Toulouse"), Toulouse, France.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** Research
**Tags:** CV
---
### [Making robotics in Peru more accessible](https://duckietown.com/nicolas-figueroa-robotics-in-peru/)
**Published:** June 20, 2025
**Author:** Federico Tani
**Excerpt:** Nicolas Figueroa, CEO of NFM Robotics and Robotics Lab, shares his vision of making robotics in Peru and Latin America accessible.
**Content:**
# Making robotics in Peru more accessible
Nicolas Figueroa, CEO of NFM Robotics and Robotics Lab, shares his vision of making robotics in Peru and Latin America accessible.
**Lima, Peru, June 2025:** Dr. Nicolas Figueroa talks with us about his goal to make teaching and learning robotics in Peru and Latin America more accessible and efficient, and especially about his mission to strengthen Peruvian national industry through robotics.
##### Quick links
- [ Nicolas Figueroa on Linkedin ](https://www.linkedin.com/in/nicolasfm/)
- [ NFM Robotics ](https://nfmrobotics.com/)
- [ Robotics Lab ](https://roboticslab.pe)
- [ University of Montpellier ](https://www.umontpellier.fr/en/)
## Bringing cutting edge robotics in Peru
##### Good morning and thank you for your time. Could you introduce yourself please?
Sure. My name is Nícolas Figueroa. I’m the general manager of NFM Robotics, and I also run a nonprofit initiative called Robotics Lab. I hold a Master’s degree in Robotics & Automation, and I recently defended my thesis, so now I’m officially a doctor!
Through Robotics Lab, we work with universities to promote robotics and robot autonomy education in Latin America, where there is still a significant gap in access to advanced robotics knowledge. I believe Duckietown offers an efficient and accessible way to help bridge this gap.
##### What can you tell us about your work?
My goal is to build a strong robotics community in Peru, and eventually throughout South America.
I work closely with university student leadership. For example, students form directive committees, presidents, vice presidents, chairs, and they organize conferences, workshops, and talks to promote robotics and robot autonomy knowledge. I maintain close contact with engineering schools in the fields of mechatronics, industrial robotics and electronics.
This connection allows me to support their efforts more effectively, even as an external partner. With NFM Robotics, we are seeing that the Peruvian industry is beginning to explore robotics, but isn’t widely adopted yet. There’s a big opportunity to offer high-level solutions, but we need more people trained in this technology.
Duckietown helps us train teams in ROS and autonomous robotics. These teams can then support industry projects.
##### So how is Duckietown useful for your work?
Considering that our target are both academic institutions for education, and industry for practical applications, I found Duckietown to be an incredible tool for introducing [autonomous robotics](https://duckietown.com/self-driving-cars-technology/). Its hands-on, accessible approach is key to closing the knowledge gap concerning robotics in Peru. When I first looked for platforms to teach autonomous robotics, I found that many options were either too expensive, had limited access, or didn’t support community engagement.
Duckietown stood out as different, it empowers learners and prioritizes impact. That’s why I knew it was the right platform to support our mission at Robotics Lab.
> Through Robotics Lab, we work with universities to promote robotics education in Latin America, where there is still a significant gap in access to advanced robotics knowledge. I believe Duckietown offers an efficient and accessible way to help bridge this gap.
>
> Nicolas Figueroa
##### What is your current focus?
Right now, we are focusing on developing robotics in Peru as a pilot project. We’ve established a presence in five Peruvian universities. But by the end of this year and early next year, we plan to expand to other countries. For example, in May, we hosted a virtual lecture series with speakers from Germany, Italy, Spain, and Estonia. It was our first step in bringing our initiative to a broader international context.
> I found Duckietown to be an incredible tool for introducing autonomous robotics. Its hands-on, accessible approach is key to closing the robotics knowledge gap.
>
> Nicolas Figueroa
##### Did Duckietown satisfy your needs?
Duckietown has become a valuable partner in our region. We’re working to bring this platform to more universities and training centers so more people can explore cutting-edge technology, reduce knowledge gaps, and prepare for Industry 4.0 challenges. We’re proud to be part of the Duckietown ecosystem and to contribute to its growth in Latin America. We hope to foster even more collaboration and opportunity for the next generation of roboticists.
##### Thank you very much for your time, any final comment?
The idea is to form a group within Robotics Lab to begin introducing autonomous robots and learning more deeply about robotic autonomy. We’re currently in discussions with some university faculties about establishing Duckietown-based laboratories, and we hope to promote our partnership with Duckietown even further.
### Learn more about Duckietown
Duckietown enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
**Categories:** People
---
### [Adapting World Models with Latent-State Dynamics Residuals](https://duckietown.com/adapting-world-models-with-latent-state-dynamics-residuals/)
**Published:** May 24, 2025
**Author:** Akshet Patel
**Excerpt:** This work uses Latent-State Dynamics Residuals for sim-to-real RL, adapting world models with minimal real data for policy transfer in Duckietown.
**Content:**
##### General Information
- **Title**: Adapting World Models with Latent-State Dynamics Residuals
- **Authors**: JB Lanier, Kyungmin Kim, Armin Karamzade, Yifei Liu, Ankita Sinha, Kat He, Davide Corsi, Roy Fox
- **Institution**: University of California Irvine, USA
- **Citation**: Lanier, J.B., Kim, K., Karamzade, A., Liu, Y., Sinha, A., He, K., Corsi, D. and Fox, R., 2025. Adapting World Models with Latent-State Dynamics Residuals. arXiv preprint arXiv:2504.02252.
[ arXiv ](https://arxiv.org/abs/2504.02252)
[ Authors ](#authors)
[ Institution ](https://www.uci.edu/)
[ Resources ](https://redraw.jblanier.net/)
# Adapting World Models with Latent-State Dynamics Residuals
Training agents for robotics applications requires a substantial amount of data, which is typically costly to collect in the real world. Running simulations is, therefore a logical approach to training agents. But to what degree do simulations provide information that correctly predicts behavior in the real world? In other words, how well do “things” learned in simulation transfer to reality? Sim2Real transfer is an exciting topic and an active area of research.
Simulation-based reinforcement learning often encounters transfer failures due to discrepancies between simulated and real-world dynamics.
This work introduces a method for **model adaptation** using **Latent-State Dynamics Residuals**, which correct transition functions in a learned latent space. A latent-variable world model, **DRAW**, is trained in simulation using **variational inference** to encode high-dimensional observations into compact **multi-categorical latent variables**.
The forward dynamics are modeled via autoregressive prediction of latent transitions. A **residual learning** function is trained on a small, **offline real-world dataset** without reward supervision to adjust the simulated dynamics. The resulting model, **ReDRAW**, modifies the forward dynamics logits using residual corrections and enables **policy training via actor-critic reinforcement learning** on **imagined rollouts**.
The reward model is reused from the simulation without retraining. To generate diverse training data, the method uses **Plan2Explore**, which promotes exploration by maximizing model uncertainty. Visual encoders trained in simulation are reused for real-world inputs through **zero-shot perception transfer**, without fine-tuning.
The approach avoids explicit observation-space correction and operates entirely in the latent space, achieving efficient **sim-to-real** policy deployment.
[ Learn about machine learning with Duckietown ](https://duckietown.com/educational-resources/)
## Highlights - adapting world models with latent-state dynamics residuals
Here is a visual tour of the sim-to-real work of the authors. For all the details, check out the [full paper](#links "Post resources").
![Architecture diagram of DRAW and ReDRAW using Latent-State Dynamics Residuals for sim-to-real RL adaptation]()
DRAW and ReDRAW World Model Training with Latent-State Dynamics Residuals
![Performance graph comparing methods using Latent-State Dynamics Residuals for sim-to-real dynamics adaptation]()
ReDRAW Performance on Dynamics Transfer in DMC Tasks
![Graphs analyzing the effect of data strategies on sim-to-real transfer using Latent-State Dynamics Residuals]()
Effect of Data Size and Source Policy on ReDRAW Transfer Performance
![Real and simulated Duckiebot environments for sim-to-real experiments using Latent-State Dynamics Residuals]()
Sim-to-Real Transfer on Duckiebot Using Latent-State Dynamics Residuals
[](https://duckietown.com/wp-content/uploads/2025/05/DRAW-and-ReDRAW-World-Model-Training-with-Latent-State-Dynamics-Residuals-png-e1749737069389.avif) DRAW and ReDRAW World Model Training with Latent-State Dynamics Residuals [](https://duckietown.com/wp-content/uploads/2025/05/ReDRAW-Performance-on-Dynamics-Transfer-in-DMC-Tasks-png-e1749737104307.avif) ReDRAW Performance on Dynamics Transfer in DMC Tasks
[](https://duckietown.com/wp-content/uploads/2025/05/Effect-of-Data-Size-and-Source-Policy-on-ReDRAW-Transfer-Performance-png-e1749737140163.avif) Effect of Data Size and Source Policy on ReDRAW Transfer Performance [](https://duckietown.com/wp-content/uploads/2025/05/Sim-to-Real-Transfer-on-Duckiebot-Using-Latent-State-Dynamics-Residuals-e1749737277282.avif) Sim-to-Real Transfer on Duckiebot Using Latent-State Dynamics Residuals
## Abstract
Here is the abstract of the work, directly in the words of the authors:
Simulation-to-reality (sim-to-real) reinforcement learning (RL) faces the critical challenge of reconciling discrepancies between simulated and real-world dynamics, which can severely degrade agent performance. A promising approach involves learning corrections to simulator forward dynamics represented as a residual error function, however this operation is impractical with high-dimensional states such as images. To overcome this, we propose ReDRAW, a latent-state autoregressive world model pretrained in simulation and calibrated to target environments through residual corrections of latent-state dynamics rather than of explicit observed states. Using this adapted world model, ReDRAW enables RL agents to be optimized with imagined rollouts under corrected dynamics and then deployed in the real world. In multiple vision-based MuJoCo domains and a physical robot visual lane-following task, ReDRAW effectively models changes to dynamics and avoids overfitting in low data regimes where traditional transfer methods fail.
## Limitations and Future Work - adapting world models with latent-state dynamics residuals
Here are the limitations and future work according to the authors of this paper:
A potential limitation with ReDRAWis that it excels at maintaining high target-environment performance over many updates because the residual avoids overfitting due to its low complexity. This suggests that only conceptually simple changes to dynamics may effectively be modeled with low amounts of data, warranting future investigation. We additionally want to explore if residual adaptation methods can be meaningfully applied to foundation world models, efficiently converting them from generators of plausible dynamics to generators of specific dynamics.
##### Did this work spark your curiosity?
Check out the follow works on machine learning with Duckietown:
- - [Sim2Real Transfer of Multi-Agent Policies for Self-Driving](https://duckietown.com/sim2real-transfer-of-multi-agent-policies-for-self-driving/ "Sim2Real Transfer of Multi-Agent Policies for Self-Driving")
- [Vision-Based DRL Autonomous Driving Agent with Sim2Real Transfer](https://duckietown.com/vision-based-drl-autonomous-driving-agent-with-sim2real-transfer-li-okhrin/ "Vision-Based DRL Autonomous Driving Agent with Sim2Real Transfer")
- [Application of PID controller and CNN to control Duckiebot robot](https://duckietown.com/pid-and-convolutional-neural-network-cnn-in-duckietown/ "Application of PID controller and CNN to control Duckiebot robot")
#### Project Authors
[  ](https://www.linkedin.com/in/jblanier/)
[JB (John Banister) Lanier](https://www.linkedin.com/in/jblanier/ "JB (John Banister) Lanier") is a Computer Science PhD Student at [UC Irvine](https://www.uci.edu/ "UC Irvine"), USA.
[  ](https://www.linkedin.com/in/kyungmin-kim-636862347/)
[Kyungmin Kim](https://www.linkedin.com/in/kyungmin-kim-636862347/ "Kyungmin Kim") is a Computer Science PhD Student at [UC Irvine](https://www.uci.edu/ "UC Irvine"), USA.
[  ](https://www.linkedin.com/in/armin-karamzade-354a78105/)
[Armin Karamzade](https://www.linkedin.com/in/armin-karamzade-354a78105/ "Armin Karamzade") is a Computer Science PhD Student at [UC Irvine](https://www.uci.edu/ "UC Irvine"), USA.
[  ](https://www.linkedin.com/in/yifei-migo-liu/)
[Yifei Liu](https://www.linkedin.com/in/yifei-migo-liu/ "Yifei Liu") is a currently an M.S. in Robotics at [Carnegie Mellon University](https://www.cmu.edu/ "Carnegie Mellon University"), USA.
[  ](https://www.linkedin.com/in/ankitasinha0811/)
[Ankita Sinha](https://www.linkedin.com/in/ankitasinha0811/ "Ankita Sinha") is currenly working as a senior LLM engineer at [NVIDIA](https://www.nvidia.com/ "NVIDIA"), USA.
Kat He was affiliated to [UC Irvine](https://www.uci.edu/ "UC Irvine"), USA during this research.
[  ](https://d-corsi.github.io/)
[Davide Corsi](https://d-corsi.github.io/ "Davide Corsi") is a Postdoctoral Researcher at [UC Irvine](https://uci.edu/ "UC Irvine"), USA.
[  ](https://www.linkedin.com/in/royf/)
[Roy Fox](https://www.linkedin.com/in/royf/ "Roy Fox") is an Assistant Professor and director of the [Intelligent Dynamics Lab (indylab)](https://indylab.org/ "Intelligent Dynamics Lab (indylab)") in the [Department of Computer Science](https://www.cs.uci.edu/ "Department of Computer Science") in the [Donald Bren School of Information & Computer Science](https://www.ics.uci.edu/ "Donald Bren School of Information & Computer Science") at the [University of California, Irvine](https://uci.edu/ "University of California, Irvine").
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** Research
**Tags:** CV, DL
---
### [Transformer Visual Control for Dynamic Obstacle Avoidance](https://duckietown.com/transformer-visual-control-for-dynamic-obstacle-avoidance/)
**Published:** May 10, 2025
**Author:** Duckietown Admin
**Excerpt:** Transformer Visual Control approach for robotic obstacle avoidance using GAS-H-Trans, validated in simulation with Duckietown environments.
**Content:**
##### General Information
- **Title**: An Adaptive Obstacle Avoidance Model for Autonomous Robots Based on Dual-Coupling Grouped Aggregation and Transformer Optimization
- **Authors**: Yuhu Tang, Ying Bai, and Qiang Chen
- **Institution**: Hefei University, China
- **Citation**: Tang, Y., Bai, Y., & Chen, Q. (2025). An Adaptive Obstacle Avoidance Model for Autonomous Robots Based on Dual-Coupling Grouped Aggregation and Transformer Optimization. Sensors, 25(6), 1839. https://doi.org/10.3390/s25061839
[ Paper ](https://www.mdpi.com/1424-8220/25/6/1839)
[ Authors ](#authors)
[ Institution ](https://www.hfuu.edu.cn/main.htm)
# Transformer Visual Control for Dynamic Obstacle Avoidance
This work details a transformer visual control approach for autonomous robotic obstacle avoidance in dynamic environments. It introduces the **GAS-H-Trans model**, which integrates a dual-coupling grouped aggregation strategy with transformer-based attention mechanisms.
Key components of the approach include grouped spatial feature aggregation, **Harris hawk optimization (HHO)** for parameter tuning, and **semantic segmentation** for real-time visual perception. The output of the segmentation is used to compute potential fields for navigation. An **artificial potential field (APF)** method, further optimized using **particle swarm optimization (PSO)**, enhances obstacle avoidance. The system was evaluated in Unity3D virtual environments and on datasets including KITTI, and ImageNet.
The model architecture improves local and global feature extraction, enabling adaptive navigation. Simulation results demonstrate that GAS-H-Trans outperforms baseline models in segmentation accuracy and avoidance reliability. The implementation uses Transformer structures, self-attention, and heuristic optimization for enhanced environmental understanding.
Experiments using Duckietown-based simulations confirm that the proposed Transformer Visual Control strategy with GAS-H-Trans significantly improves obstacle avoidance reliability with respect to typical approaches.
[ Learn about machine learning with Duckietown materials ](https://duckietown.com/educational-resources/)
## Highlights - Transformer Visual Control for Dynamic Obstacle Avoidance
Here is a visual tour of this work. For all the details, check out the [full paper](#links "Post resources").
![GAS-H-Trans Transformer Visual Control system architecture for Duckietown robotic navigation]()
Figure 1. GAS-H-Trans Framework Architecture
![Transformer Visual Control model architecture using GAS-H-Trans and grouped aggregation in Duckietown simulation]()
Figure 2. Detailed Architecture of GAS-H-Trans Model
![PSO-optimized artificial potential field for Transformer Visual Control in Duckietown simulation]()
Figure 3. Artificial Potential Field Optimization Flow
![Transformer Visual Control model accuracy chart in Duckietown obstacle avoidance task]()
Figure 4. Model Accuracy Comparison Across Epochs
![Transformer Visual Control segmentation performance in Duckietown using ResNet50 and GAS-H-Trans]()
Figure 5. Segmentation Output Comparison
![Swin Transformer comparison with GAS-H-Trans in Transformer Visual Control on Duckietown data]()
Figure 6. Swin Transformer vs GAS-H-Trans Performance
![Duckietown virtual environment dataset creation for Transformer Visual Control experiments]()
Figure 7. Dataset Creation Process
![GAS-H-Trans Transformer Visual Control segmentation accuracy in Duckietown virtual dataset]()
Figure 8. Image Segmentation Accuracy in Duckietown Scene
![Segmentation output of Transformer Visual Control with JSON mapping in Duckietown task]()
Figure 9. Segmentation Mask and Data Structure
![Transformer Visual Control path planning with GAS-H-Trans and PSO in Duckietown]()
Figure 10. Obstacle Avoidance in Duckietown Environment
![Comparison of traditional and PSO-optimized APF for Transformer Visual Control in Duckietown]()
Figure 11. Effect of PSO Optimization on APF
![Enhanced Transformer Visual Control robot navigation path in Duckietown simulation]()
Figure 12. Optimized Obstacle Avoidance Path
![Transformer Visual Control success rate with traditional vs PSO-optimized APF in Duckietown]()
Figure 13. Obstacle Avoidance Success Rate
[](https://duckietown.com/wp-content/uploads/2025/05/GAS-H-Trans-Framework-Architecture-png.avif) Figure 1. GAS-H-Trans Framework Architecture [](https://duckietown.com/wp-content/uploads/2025/05/Detailed-Architecture-of-GAS-H-Trans-Model-png.avif) Figure 2. Detailed Architecture of GAS-H-Trans Model [](https://duckietown.com/wp-content/uploads/2025/05/Artificial-Potential-Field-Optimization-Flow-png.avif) Figure 3. Artificial Potential Field Optimization Flow
[](https://duckietown.com/wp-content/uploads/2025/05/Model-Accuracy-Comparison-Across-Epochs-png.avif) Figure 4. Model Accuracy Comparison Across Epochs [](https://duckietown.com/wp-content/uploads/2025/05/Segmentation-Output-Comparison.avif) Figure 5. Segmentation Output Comparison [](https://duckietown.com/wp-content/uploads/2025/05/Swin-Transformer-vs-GAS-H-Trans-Performance-png.avif) Figure 6. Swin Transformer vs GAS-H-Trans Performance
[](https://duckietown.com/wp-content/uploads/2025/05/Dataset-Creation-Process-png.avif) Figure 7. Dataset Creation Process [](https://duckietown.com/wp-content/uploads/2025/05/Image-Segmentation-Accuracy-in-Duckietown-Scene-png.avif) Figure 8. Image Segmentation Accuracy in Duckietown Scene [](https://duckietown.com/wp-content/uploads/2025/05/Segmentation-Mask-and-Data-Structure-png.avif) Figure 9. Segmentation Mask and Data Structure
[](https://duckietown.com/wp-content/uploads/2025/05/Obstacle-Avoidance-in-Duckietown-Environment-png.avif) Figure 10. Obstacle Avoidance in Duckietown Environment [](https://duckietown.com/wp-content/uploads/2025/05/Effect-of-PSO-Optimization-on-APF-png.avif) Figure 11. Effect of PSO Optimization on APF [](https://duckietown.com/wp-content/uploads/2025/05/Optimized-Obstacle-Avoidance-Path-png.avif) Figure 12. Optimized Obstacle Avoidance Path
[](https://duckietown.com/wp-content/uploads/2025/05/Obstacle-Avoidance-Success-Rate-png.avif) Figure 13. Obstacle Avoidance Success Rate
## Abstract
In the author’s words:
Accurate obstacle recognition and avoidance are critical for ensuring the safety and operational efficiency of autonomous robots in dynamic and complex environments. Despite significant advances in deep-learning techniques in these areas, their adaptability in dynamic and complex environments remains a challenge. To address these challenges, we propose an improved Transformer-based architecture, GAS-H-Trans.
This approach uses a grouped aggregation strategy to improve the robot’s semantic understanding of the environment and enhance the accuracy of its obstacle avoidance strategy. This method employs a Transformer-based dual-coupling grouped aggregation strategy to optimize feature extraction and improve global feature representation, allowing the model to capture both local and long-range dependencies.
The Harris hawk optimization (HHO) algorithm is used for hyperparameter tuning, further improving model performance. A key innovation of applying the GAS-H-Trans model to obstacle avoidance tasks is the implementation of a secondary precise image segmentation strategy. By placing observation points near critical obstacles, this strategy refines obstacle recognition, thus improving segmentation accuracy and flexibility in dynamic motion planning. The particle swarm optimization (PSO) algorithm is incorporated to optimize the attractive and repulsive gain coefficients of the artificial potential field (APF) methods.
This approach mitigates local minima issues and enhances the global stability of obstacle avoidance. Comprehensive experiments are conducted using multiple publicly available datasets and the Unity3D virtual robot environment. The results show that GAS-H-Trans significantly outperforms existing baseline models in image segmentation tasks, achieving the highest mIoU (85.2%). In virtual environment obstacle avoidance tasks, the GAS-H-Trans + PSO-optimized APF framework achieves an impressive obstacle avoidance success rate of 93.6%. These results demonstrate that the proposed approach provides superior performance in dynamic motion planning, offering a promising solution for real-world autonomous navigation applications.
## Conclusion - Transformer Visual Control for Dynamic Obstacle Avoidance
Here is the author’s summary and overview of lessons learned from this work:
In this study, we proposed the GAS-H-Trans framework for image segmentation and dynamic obstacle avoidance in autonomous robots. The key contributions are summarized as follows. **(1) Dual-coupling grouped aggregation strategy:** A Transformer-based dualcoupling grouped aggregation method optimizes feature extraction and enhances global feature representation, thereby improving the model’s perception performance in dynamic motion planning. **(2) Harris hawk optimization (HHO):** The integration of the HHO algorithm into the GAS-Trans framework optimizes the number of Transformer layers and iterations, improving model accuracy and reducing computational costs. **(3) PSOoptimized artificial potential field (APF):** We integrated the PSO algorithm with APF to optimize the attractive and repulsive gain coefficients, addressing local minima issues and enhancing the global stability of the obstacle avoidance system.
This study also proposes a secondary precise image segmentation strategy. By setting the observation points near critical obstacles for fine-tuned segmentation, the flexibility and accuracy of the segmentation model’s environmental perception are effectively enhanced, thereby improving the robot’s obstacle avoidance capabilities.
Through the integration of PSO-optimized APF with image segmentation, the GAS-HTrans + PSO-optimized APF framework demonstrated significant improvements in obstacle avoidance. In the experimental validation of this study, the obstacles remained static throughout the navigation process. Using this method, the autonomous robot dynamically adjusted its obstacle avoidance trajectory based on segmented environmental features. This integration significantly enhanced environmental perception capabilities and the accuracy of obstacle avoidance decisions, enabling more efficient navigation in static obstacle environments.
Extensive experiments on publicly available datasets (Duckiebot, KITTI, ImageNet) and in the Unity3D virtual robot environment validate the effectiveness of the proposed framework. The GAS-H-Trans framework outperformed traditional models in image segmentation tasks, achieving the highest mIoU of 85.2%. Furthermore, in virtual obstacle avoidance experiments, the GAS-H-Trans + PSO-optimized APF framework achieved an obstacle avoidance success rate of 93.6%.
These results effectively validate the proposed strategy, which combines secondary image segmentation from GAS-H-Trans with the PSO-optimized APF method, significantly improving obstacle avoidance performance in dynamic motion planning. Additionally, the GAS-H-Trans framework has the potential to be extended to fully dynamic environments by incorporating real-time object tracking and adaptive obstacle modeling. However, some limitations exist. The majority of the experiments were conducted in simulated environments, and future research will focus on validating the framework in real-world scenarios and improving real-time performance.
Additionally, the integration of multi-modal sensor data (such as LiDAR and ultrasonic sensors) will be an important direction for future work to further enhance environmental perception and robustness.
In conclusion, the new framework offers an innovative solution for autonomous robot obstacle avoidance in dynamic motion planning. Its powerful environmental perception and obstacle avoidance performance demonstrate significant potential for practical applications. With further optimization and real-world validation, this framework will play a crucial role in the future development of autonomous navigation and robotics technology.
##### Did this work spark your curiosity?
Check out the follow works on machine learning with Duckietown:
- - [Semantic Image Segmentation Methods in the Duckietown Project](https://duckietown.com/semantic-image-segmentation-methods-in-duckietown/ "Semantic Image Segmentation Methods in the Duckietown Project")
- [Agent-Based Autonomous Robotic System Using Deep Reinforcement and Transfer Learning](https://duckietown.com/deep-reinforcement-and-transfer-learning-for-robot-autonomy/ "Agent-Based Autonomous Robotic System Using Deep Reinforcement and Transfer Learning")
- [Application of PID controller and CNN to control Duckiebot robot](https://duckietown.com/pid-and-convolutional-neural-network-cnn-in-duckietown/ "Application of PID controller and CNN to control Duckiebot robot")
#### Project Authors
Yuhu Tang is affiliated with the School of Artificial Intelligence and Big Data, [Hefei University](https://www.hfuu.edu.cn/main.htm "Hefei University"), Hefei 230601, China.
Ying Bai is affiliated with the School of Artificial Intelligence and Big Data, [Hefei University](https://www.hfuu.edu.cn/main.htm "Hefei University"), Hefei 230601, China.
[  ](https://ieeexplore.ieee.org/author/37061090100)
[Qiang Chen](https://ieeexplore.ieee.org/author/37061090100 "Qiang Chen") is affiliated with School of Electrical Engineering and Automation
National and Local Joint Engineering Laboratory for Renewable Energy Access to Grid Technology, Hefei University of Technology, Hefei, China, [Hefei University](https://www.hfuu.edu.cn/main.htm "Hefei University"), Hefei 230601, China.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** Research
**Tags:** CV, DL
---
### [VAE-Based Out-of-Distribution Detectors for Embedded Systems](https://duckietown.com/vae-based-out-of-distribution-detectors-for-embedded-systems/)
**Published:** April 11, 2025
**Author:** Duckietown Admin
**Excerpt:** Compressing VAE-based OOD detectors for real-time deployment in Duckietown with pruning, quantization, and knowledge distillation.
**Content:**
##### General Information
- **Title**: Compressing VAE-Based Out-of-Distribution Detectors for Embedded Deployment
- **Authors**: Aditya Bansal, Michael Yuhas, Arvind Easwaran
- **Institution**: Nanyang Technological University, SIngapore
- **Citation**: Bansal, A., Yuhas, M., & Easwaran, A. (2024). Compressing VAE-Based Out-of-Distribution Detectors for Embedded Deployment. In Proceedings - 2024 IEEE 30th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2024 (pp. 37-42).
[ Paper ](https://nanyangtechnologicaluniv.demo.elsevierpure.com/en/publications/compressing-vae-based-out-of-distribution-detectors-for-embedded-)
[ Authors ](#authors)
[ Institution ](https://www.ntu.edu.sg/)
# VAE-Based Out-of-Distribution Detectors for Embedded Systems
Out-of-distribution (OOD) detection is essential for maintaining safety in machine learning systems, especially those operating in the real world. It helps identify inputs that differ significantly from the training data, which could lead to unexpected or unsafe behavior.
Variational Autoencoders (VAEs) are neural networks that compress input data into a smaller latent space (a compact set of features) and reconstructs the input from this compressed version.
In OOD detection, if the reconstruction fails or doesn’t fit the expected latent space, the input is flagged as unfamiliar, i.e., out-of-distribution. While VAEs are effective, they are computationally expensive, making them hard to deploy on small, embedded devices like Duckiebots.
To solve this challenge, building upon previous work ([Embedded Out-of-Distribution Detection on an Autonomous Robot Platform](https://duckietown.com/embedded-out-of-distribution-detection-in-duckietown/)), the researchers applied three model compression techniques:
- **Pruning**: Removes low-importance weights or neurons to shrink and speed up the model.
- **Knowledge distillation**: Trains a smaller “student” model to mimic a larger “teacher” model.
- **Quantization**: Lowers numerical precision (e.g., from 32-bit to 8-bit) to save memory and improve speed.
Two VAE-based OOD detectors were evaluated:
- **β-VAE**: A variant of VAE that learns more interpretable latent features (controlled by a parameter called β).
- **Optical Flow Detector**: Analyzes how pixels move across video frames to detect unusual motion.
Both models were trained and tested using data collected in Duckietown, and the models were measured using Area under the Receiver Operating Characteristic Curve ([AUROC](https://h2o.ai/wiki/auc-roc/)), which shows how well the model separates known from unknown inputs, memory footprint, and execution latency. The compressed models achieved faster inference times, smaller memory usage, and only minor drops in detection accuracy.
[ Learn about machine learning with Duckietown materials ](https://duckietown.com/educational-resources/)
[ Check out this research on Variational Autoencoder for autonomous driving in Duckietown ](https://duckietown.com/variational-autoencoder-for-autonomous-driving-in-duckietown/)
## Highlights - VAE-Based Out-of-Distribution Detectors for Embedded Systems
Here is a visual tour of the work of the authors. For all the details, check out the [full paper](#links "Post resources").
![Visual control architecture in Duckietown showing VAE OOD compression methodology]()
Figure 1. Design Methodology for VAE Compression
![Visual control models in Duckietown showing β-VAE and Optical Flow OOD detectors]()
Figure 2. OOD Detector Block Diagrams
![Visual control output samples in Duckietown using VAE reconstructions]()
Figure 3. Reconstructed Images from β-VAE
![Visual control metrics in Duckietown with β-VAE under varying sparsity]()
Figure 4. β-VAE AUROC vs. Sparsity
![Visual control performance in Duckietown using various compression methods]()
Figure 5. Compression Impact on β-VAE
![Visual control optimization in Duckietown using compressed optical flow models]()
Figure 6. Knowledge Distillation Effects on Optical Flow Model
![Visual control performance degradation in Duckietown under extreme sparsity]()
Figure 7. Sparsity Impact on Optical Flow Models
[](https://duckietown.com/wp-content/uploads/2025/04/Design-Methodology-for-VAE-Compression-png.avif) Figure 1. Design Methodology for VAE Compression [](https://duckietown.com/wp-content/uploads/2025/04/OOD-Detector-Block-Diagrams-png.avif) Figure 2. OOD Detector Block Diagrams [](https://duckietown.com/wp-content/uploads/2025/04/Reconstructed-Images-from-β-VAE.avif) Figure 3. Reconstructed Images from β-VAE
[](https://duckietown.com/wp-content/uploads/2025/04/β-VAE-AUROC-vs.-Sparsity-png.avif) Figure 4. β-VAE AUROC vs. Sparsity [](https://duckietown.com/wp-content/uploads/2025/04/Compression-Impact-on-β-VAE-png.avif) Figure 5. Compression Impact on β-VAE [](https://duckietown.com/wp-content/uploads/2025/04/Knowledge-Distillation-Effects-on-Optical-Flow-Model-png.avif) Figure 6. Knowledge Distillation Effects on Optical Flow Model
[](https://duckietown.com/wp-content/uploads/2025/04/Sparsity-Impact-on-Optical-Flow-Models.avif) Figure 7. Sparsity Impact on Optical Flow Models
## Abstract
In the author’s words:
Out-of-distribution (OOD) detectors can act as safety monitors in embedded cyber-physical systems by identifying samples outside a machine learning model’s training distribution to prevent potentially unsafe actions. However, OOD detectors are often implemented using deep neural networks, which makes it difficult to meet real-time deadlines on embedded systems with memory and power constraints. We consider the class of variational autoencoder (VAE) based OOD detectors where OOD detection is performed in latent space, and apply quantization, pruning, and knowledge distillation.
These techniques have been explored for other deep models, but no work has considered their combined effect on latent space OOD detection. While these techniques increase the VAE’s test loss, this does not correspond to a proportional decrease in OOD detection performance and we leverage this to develop lean OOD detectors capable of real-time inference on embedded CPUs and GPUs. We propose a design methodology that combines all three compression techniques and yields a significant decrease in memory and execution time while maintaining AUROC for a given OOD detector.
We demonstrate this methodology with two existing OOD detectors on a Jetson Nano and reduce GPU and CPU inference time by 20% and 28% respectively while keeping AUROC within 5% of the baseline.
## Conclusion - VAE-Based Out-of-Distribution Detectors for Embedded Systems
Here are the conclusions from the author of this paper:
We explored different neural network compression techniques on β-VAE and optical flow OOD detectors using a mobile robot powered by a Jetson Nano. Based on our analysis of results for quantization, knowledge distillation, and pruning, we proposed a design strategy to find the model with the best execution time and memory usage while maintaining some accuracy metric for a given VAE-based OOD detector. We successfully demonstrated this methodology on an optical flow OOD detector and showed that our methodology’s ability to aggressively prune and compress a model is due to the unique attributes of VAE-based OOD detection.
Despite our methodology’s good performance, it requires access to OOD samples at design time to act as a crossvalidation set. In our case study, we assume OOD samples arise from a particular generating distribution, but this may not be the case in general. Furthermore, it only guides the search for a faster architecture, but does not guarantee the optimum result. Nevertheless, we believe having a design methodology that combines quantization, knowledge distillation, and pruning allows engineers to exploit the combined powers of these techniques instead of considering them individually.
#### Project Authors
[  ](https://www.linkedin.com/in/aditya-bansal-10/)
[Aditya Bansal](https://www.linkedin.com/in/aditya-bansal-10/) is currently working as a Machine Learning Engineer at [Adobe](https://www.adobe.com/), United States.
[  ](https://www.linkedin.com/in/michael-yuhas-091b347a/?originalSubdomain=sg)
[Michael Yuhas](https://www.linkedin.com/in/michael-yuhas-091b347a/?originalSubdomain=sg) is currenly working as a Research Assistant at [Nanyang Technological University](https://www.ntu.edu.sg/), Singapore.
[  ](https://www.linkedin.com/in/arvind-easwaran-066544292/?originalSubdomain=sg)
[Arvind Easwaran](https://www.linkedin.com/in/arvind-easwaran-066544292/?originalSubdomain=sg) is an Associate Professor at [Nanyang Technological University](https://www.ntu.edu.sg/), Singapore.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** Research
**Tags:** CV, DL
---
### [Embedded Out-of-Distribution Detection in Duckietown](https://duckietown.com/embedded-out-of-distribution-detection-in-duckietown/)
**Published:** January 11, 2025
**Author:** Duckietown Admin
**Excerpt:** This research explores Embedded Out-of-Distribution (OOD) Detection in Duckietown, resulting in 87.5% succesful detections for real-time autonomous systems.
**Content:**
##### General Information
- **Title**: Embedded Out-of-Distribution Detection on an Autonomous Robot Platform
- **Authors**: Michael Yuhas, Yeli Feng, Daniel Jun Xian Ng, Zahra Rahiminasab, Arvind Easwaran
- **Institution**: Nanyang Technological University, Singapore
- **Citation**: Yuhas, M., Feng, Y., Ng, D.J.X., Rahiminasab, Z. and Easwaran, A., 2021, May. Embedded out-of-distribution detection on an autonomous robot platform. In Proceedings of the Workshop on Design Automation for CPS and IoT (pp. 13-18).
[ Paper ](https://dl.acm.org/doi/10.1145/3445034.3460509)
[ Authors ](#authors)
[ Institution ](https://www.ntu.edu.sg/)
# Embedded Out-of-Distribution Detection in Duckietown
The project “embedded out-of-distribution detection (OOD) Detection on an Autonomous Robot Platform” focuses on safety in Duckietown by implementing real-time OOD detection on the Duckiebots. The concept involves using a machine learning-based OOD detector, specifically a β-Variational Autoencoder (β-VAE), to identify test inputs that deviate from the training data’s distribution. Such inputs can lead to unreliable behavior in machine learning systems, critical for safety in autonomous platforms like the Duckiebot.
Key aspects of the project include:
- **Integration:** The β-VAE OOD detector is integrated with the Duckiebot’s ROS-based architecture, alongside lane-following and motor control modules.
- **Emergency Braking:** An emergency braking mechanism halts the Duckiebot when OOD inputs are detected, ensuring safety during operation.
- **Evaluation:** Performance was evaluated in scenarios where the Duckiebot navigated a track and avoided obstacles. The system achieved an 87.5% success rate in emergency stops.
This work demonstrates a method to mitigate safety risks in autonomous robotics. By providing a framework for OOD detection on low-cost platforms, the project contributes to the broader applicability of safe machine learning in cyber-physical systems.
[ Learn about machine learning with Duckietown materials ](https://duckietown.com/educational-resources/#machine-learning)
## Highlights - Embedded Out-of-Distribution Detection in Duckietown
Here is a visual tour of the work of the authors. For all the details, check out the [full paper](#links "Post resources").
![Figure 1. Duckietown DB18 Robot.]()
Figure 1. Duckietown DB18 Robot.
![Diagram showing the Duckietown software stack integrated with ROS packages, illustrating the architecture and components developed for this research.]()
Figure 2. Duckietown Software Stack with Integrated ROS Packages.
![A block diagram illustrating the Embedded Out-of-Distribution Detection architecture built on the existing Duckietown framework, showing components and data flow.]()
Figure 3. OOD Detection Architecture Using Duckietown Framework.
![A set of sample images showing in-distribution data from Duckiebot and nuScenes on the top row, and OOD obstacle images at varying distances on the bottom row.]()
Figure 4. Sample In-Distribution and OOD Images from Experiment.
![The setup for the emergency braking experiment, showing a Duckiebot moving at a constant velocity toward a stationary obstacle within its risk zone.]()
Figure 5. Emergency Braking Experiment Setup.
![A plot showing the distribution of OOD scores as a function of distance from the starting position, with lines representing different test runs and stopping distances.]()
Figure 6. Distribution of OOD Scores by Distance and Stopping Performance.
![A violin plot showing the distribution of sub-task execution times across all test runs, with variations in time visually represented by the plot's shape.]()
Figure 7. Distribution of Sub-Task Execution Times for Test Runs.
![Boxplots with confidence intervals showing the distribution of projected stopping distances for different OOD detection thresholds, with medians marked.]()
Figure 8. Projected Stopping Distances for Varying OOD Detection Thresholds.
## Abstract
In the author’s words:
Machine learning (ML) is actively finding its way into modern cyber-physical systems (CPS), many of which are safety-critical real-time systems. It is well known that ML outputs are not reliable when testing data are novel with regards to model training and validation data, i.e., out-of-distribution (OOD) test data. We implement an unsupervised deep neural network-based OOD detector on a real-time embedded autonomous Duckiebot and evaluate detection performance. Our OOD detector produces a success rate of 87.5% for emergency stopping a Duckiebot on a braking test bed we designed. We also provide case analysis on computing resource challenges specific to the Robot Operating System (ROS) middleware on the Duckiebot.
## Conclusion - Embedded Out-of-Distribution Detection in Duckietown
Here are the conclusions from the author of this paper:
“We successfully demonstrated that the 𝛽-VAE OOD detection algorithm could run on an embedded platform and provides a safety check on the control of an autonomous robot. We also showed that performance is dependent on real-time performance of the embedded system, particularly the OOD detector execution time. Lastly, we showed that there is a trade-off involved in choosing an OOD detection threshold; a smaller threshold value increases the average stopping distance from an obstacle, but leads to an increase in false positives.
This work also generates new questions that we hope to investigate in the future. The system architecture demonstrated in this paper was not utilizing a real-time OS and did not take advantage of technologies such as GPUs or TPUs, which are now becoming common on embedded systems. There is still much work that can be done to optimize process scheduling and resource utilization while maintaining the goal of using low-cost, off-the-shelf hardware and open-source software. Understanding what quality of service can be provided by a system with these constraints and whether it suffices for reliable operations of OOD detection algorithms is an ongoing research theme.
From the OOD detection perspective, we would like to run additional OOD detection algorithms on the same architecture and compare performance in terms of accuracy and computational efficiency. We would also like to develop a more comprehensive set of test scenarios to serve as a benchmark for OOD detection on embedded systems. These should include dynamic as well as static obstacles, operation in various environments and lighting conditions, and OOD scenarios that occur while the robot is performing more complex tasks like navigating corners, intersections, or merging with other traffic.
Demonstrating OOD detection on the Duckietown platform opens the door for more embedded applications of OOD detectors. This will serve to better evaluate their usefulness as a tool to enhance the safety of ML systems deployed as part of critical CPS.”
##### Did this work spark your curiosity?
The authors followed up with additional research on this very topic:
- [Compressing VAE-Based Out-of-Distribution Detectors for Embedded Deployment](https://duckietown.com/vae-based-out-of-distribution-detectors-for-embedded-systems/ "Compressing VAE-Based Out-of-Distribution Detectors for Embedded Deployment")
Other works using variational autoencoders with Duckietown:
- [Learning to Drive with Reinforcement Learning and Variational Autoencoders](https://duckietown.com/variational-autoencoder-for-autonomous-driving-in-duckietown/ "Learning to Drive with Reinforcement Learning and Variational Autoencoders")
#### Project Authors
[  ](https://www.linkedin.com/in/michael-yuhas-091b347a/?originalSubdomain=sg)
[Michael Yuhas](https://www.linkedin.com/in/michael-yuhas-091b347a/?originalSubdomain=sg) is currenly working as a Research Assistant and pursuing his PhD at the [Nanyang Technological University](https://www.ntu.edu.sg/), Singapore.
[  ](https://www.linkedin.com/in/yelifeng/?originalSubdomain=sg)
[Yeli Feng](https://www.linkedin.com/in/yelifeng/?originalSubdomain=sg) is currenly working as a Lead Data Scientist at [Amplify Health](https://www.amplifyhealth.com/), Singapore.
[  ](https://www.linkedin.com/in/daniel-jun-xian-ng-51236a146/?originalSubdomain=sg)
[Daniel Jun Xian Ng](https://www.linkedin.com/in/daniel-jun-xian-ng-51236a146/?originalSubdomain=sg) is currenly working as a Mobile Robot Software Engineer at the [Hyundai Motor Group Innovation Center Singapore (HMGICS)](https://www.hyundai.com/sg/home), Singapore.
[  ](https://www.linkedin.com/in/zahra-rahiminasab-ph-d-6b82a0178/?originalSubdomain=sg)
[Zahra Rahiminasab](https://www.linkedin.com/in/zahra-rahiminasab-ph-d-6b82a0178/?originalSubdomain=sg) is currenly working as a Postdoctoral Researcher at [Aalto University](https://www.aalto.fi/en), Finland.
[  ](https://www.linkedin.com/in/arvind-easwaran-066544292/?originalSubdomain=sg)
[Arvind Easwaran](https://www.linkedin.com/in/arvind-easwaran-066544292/?originalSubdomain=sg) is currenly working as an Associate Professor at the [Nanyang Technological University](https://www.ntu.edu.sg/), Singapore.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
---
### [Semantic Image Segmentation Methods in Duckietown](https://duckietown.com/semantic-image-segmentation-methods-in-duckietown/)
**Published:** March 27, 2025
**Author:** Duckietown Admin
**Excerpt:** Semantic image segmentation studies the problem of giving meaning to pixels. Authors explore deep learning models: SegNet, U-Net, FC-DenseNet, and DeepLab-v3.
**Content:**
##### General Information
- **Title**: Semantic Image Segmentation Methods in the Duckietown Project
- **Authors**: Kristina S. Lanchukovskaya; Dasha E. Shabalina; Tatiana V. Liakh
- **Institution**: Novosibirsk State University, The Russian Federation
- **Citation**: Lanchukovskaya, K.S., Shabalina, D.E. and Liakh, T.V., 2022, June. Semantic image segmentation methods in the duckietown project. In 2022 IEEE 23rd International Conference of Young Professionals in Electron Devices and Materials (EDM) (pp. 611-617). IEEE.
[ Paper ](https://ieeexplore.ieee.org/document/9855168)
[ Authors ](#authors)
[ Institution ](https://english.nsu.ru/)
# Semantic Image Segmentation Methods in Duckietown
In Duckietown, where self-driving agents (i.e., Duckiebots) operate in structured environments, segmentation is essential for lane detection, object recognition, and obstacle avoidance. Semantic Image Segmentation assigns a class label to each pixel in an image, allowing autonomous systems to interpret their surroundings.
This research evaluates four deep learning models – [SegNet](), [U-Net](https://en.wikipedia.org/wiki/U-Net), [FC-DenseNet](https://arxiv.org/abs/1611.09326), and [DeepLab-v3](https://paperswithcode.com/method/deeplabv3) by comparing their efficiency, accuracy, and real-time applicability. Understanding the trade-offs between these models helps optimize perception for Duckiebots navigating the Duckietown.
These models rely on [Convolutional Neural Networks](https://en.wikipedia.org/wiki/Convolutional_neural_network "Convolutional Neural Networks Wikipedia") (CNNs) to extract hierarchical features. SegNet prioritizes memory efficiency, U-Net incorporates skip connections for improved localization, FC-DenseNet enhances feature reuse through dense connectivity, and DeepLab-v3 captures multi-scale context with atrous spatial pyramid pooling. Each model presents a balance between computational cost and segmentation accuracy, influencing its suitability for embedded systems like Duckiebots.
Implementing semantic segmentation in Duckietown enhances autonomy by enabling self-driving agents to interpret complex visual inputs. The selection of an appropriate segmentation model depends on processing constraints and real-time performance needs. By integrating optimized segmentation techniques, Duckiebots improve decision-making in structured environments.
[ Learn about machine learning with Duckietown materials ](https://duckietown.com/educational-resources/)
## Highlights - Semantic Image Segmentation Methods in Duckietown
Here is a visual tour of the work of the authors. For all the details, check out the [full paper](#links "Post resources").
[](https://duckietown.com/wp-content/uploads/2025/03/SegNet-Architecture-for-Semantic-Image-Segmentation-png.avif) Figure 1. SegNet Architecture for Semantic Image Segmentation [](https://duckietown.com/wp-content/uploads/2025/03/U-Net-Architecture-for-Semantic-Image-Segmentation-png.avif) Figure 2. U-Net Architecture for Semantic Image Segmentation
[](https://duckietown.com/wp-content/uploads/2025/03/FC-DenseNet-for-Semantic-Image-Segmentation-png.avif) Figure 3. FC-DenseNet for Semantic Image Segmentation [](https://duckietown.com/wp-content/uploads/2025/03/DeepLab-v3-for-Semantic-Image-Segmentation-png.avif) Figure 4. DeepLab-v3 for Semantic Image Segmentation
## Abstract
In the author’s words:
The article focuses on evaluation of the applicability of existing semantic segmentation algorithms for the Duckietown simulator. Duckietown is an open research project in the field of autonomously controlled robots. The article explores classical semantic image segmentation algorithms. Their analysis for applicability in Duckietown is carried out.
With the help of them, we want to make a dataset for training neural networks. The following was investigated: edge-detection techniques, threshold algorithms, region growing, segmentation algorithms based on clustering, neural networks. The article also reviewed networks designed for semantic image segmentation and machine learning frameworks, taking into account all the limitations of the Duckietown simulator.
Experiments were conducted to evaluate the accuracy of semantic segmentation algorithms on such classes of Duckietown objects as road and background. Based on the results of the analysis, region growing algorithms and clustering algorithms were selected and implemented.
Experiments were conducted to evaluate the accuracy on such classes of Duckietown objects as road, background and traffic signs. After evaluating the accuracy of the algorithms considered, it was decided to use Color segmentation, Mean Shift, Thresholding algorithms and Segmentation of signs by April-tag for image preprocessing. For neural networks, experiments were conducted to evaluate the accuracy of semantic segmentation algorithms on such classes of Duckietown objects as road and background. After evaluating the accuracy of the algorithms considered, it was decided to select the DeepLab-v3 neural network. Separate module was created for semantic image segmentation in Duckietown.
## Conclusion - Semantic Image Segmentation Methods in Duckietown
Here are the conclusions from the author of this paper:
The article analyzes the applicability of semantic segmentation algorithms in the Duckietown simulator, which simulates autopilot robots in an urban environment.
It was found that methods based on classical computer vision algorithms are inferior to methods based on neural networks in terms of stability, segmentation accuracy and speed of operation. It was proposed to use classical computer vision algorithms for marking images and preparing datasets and neural networks for segmentation on robots.
CV algorithms taking into account the features of the Duckietown simulator. Thus, classical computer vision algorithms, such as area-building algorithms and clustering algorithms, were chosen for image preprocessing. OpenCV and Scikit-image libraries were selected for the experiment. The best result during the testing was obtained using MeanShift and cv2.threshold together, and road signs were segmented most successfully using April tag.
Also, after testing the selected neural networks, it was decided to select the DeepLab-v3 neural network as an adapted semantic segmentation algorithm for the Duckietown simulator. After testing the trained DeepLab-v3 neural network model on Duckiebot, a separate module for semantic image segmentation was created in the Duckietown open research project. In the future, it is planned to add such classes of Duckietown objects as a duck in the role of a pedestrian, road markings (red, yellow, white) and Duckiebot.
#### Project Authors
[Kristina S. Lanchukovskaya](https://ieeexplore.ieee.org/author/37089493784) is affiliated with the department of IT, [Novosibirsk State University](https://english.nsu.ru/), Novosibirsk, Russia.
[Dasha E. Shabalina](https://ieeexplore.ieee.org/author/37089494405) is affiliated with the department of IT, [Novosibirsk State University](https://english.nsu.ru/), Novosibirsk, Russia.
[  ](https://ieeexplore.ieee.org/author/37086855118)
[Tatiana V. Liakh](https://ieeexplore.ieee.org/author/37086855118) is a Senior Lecturer at the Department of Computer Science, Electrical and Space Engineering, [Novosibirsk State University](https://english.nsu.ru/), Novosibirsk, Russia.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** Research
**Tags:** CV, DL, semantic segmentation
---
### [City Rescue: Autonomous Recovery System for Duckiebots](https://duckietown.com/city-rescue-autonomous-recovery-system-for-duckiebots/)
**Published:** March 23, 2025
**Author:** Duckietown Admin
**Excerpt:** Learn how to transform Duckietown in a smart city performing autonomous recovery of distressed Duckiebots, through vehicle to infrastructure (v2i) interactions.
**Content:**
# City Rescue: Autonomous Recovery System for Duckiebots
##### Project Resources
- **Objective**: Develop an autonomous recovery system to detect distressed Duckiebots in Duckietown and rescue them back to the road.
- **Approach**: Utilize a Duckietown infratructure-based "real-time" localization system integrated with a server-based rescue mechanism to monitor, classify distress, and execute recovery maneuvers.
- **Authors**: Carl Biagosch, Shengjie Hu, Martin Ziran Xu
[ Report ](#thesis)
[ University ](https://ethz.ch/en.html)
[ Authors ](#authors)
[ Code ](https://github.com/duckietown-ethz/proj-cityrescue)
## Project highlights
[](https://duckietown.com/wp-content/uploads/2025/03/Software-Architecture-Overview-png.avif) Figure 1. Software Architecture Overview [](https://duckietown.com/wp-content/uploads/2025/03/Server-Side-Software-Architecture-png.avif) Figure 2. Server-Side Software Architecture [](https://duckietown.com/wp-content/uploads/2025/03/Simple-Localization-vs.-CSLAM-png.avif) Figure 3. Simple Localization vs. CSLAM
[](https://duckietown.com/wp-content/uploads/2025/03/Map-Visualization-of-AMOD-K31-png.avif) Figure 4. Map Visualization of AMOD-K31 [](https://duckietown.com/wp-content/uploads/2025/03/3-Way-Intersection-Path-Planning-png.avif) Figure 5. 3-Way Intersection Path Planning [](https://duckietown.com/wp-content/uploads/2025/03/4-Way-Intersection-Path-Planning-png.avif) Figure 6. 4-Way Intersection Path Planning
## City rescue: autonomous recovery system for Duckiebots - the objectives
Would it not be desirable to have the city we drive in monitor our vehicle, as a guardian angel ready to intervene in case of distress offering autonomous recovery services?
The project, “City Rescue” is a first step towards enabling a continuous monitoring system from traffic lights and watchtowers, smart infrastructure in Duckietown, aimed at localization and communicating with Duckiebots as they autonomously operate in town.
Despite the robust autonomy algorithms guiding the behaviors of Duckietown in Duckietowns, distress situations such as lane departures, crashes, or stoppages, might happen. In these cases human intervention is often necessary to reset experiments.
This project introduces an automated monitoring and rescue system that identifies distressed agents, classifies their distress state, and calculates and communicates corrective actions to restore Duckiebots to normal operation.
The City-Rescue project incorporates several key components to achieve autonomous monitoring and recovery of distressed Duckiebots:
- **Distress detection**: classifies failure states such as lane departure, collision, and immobility using real-time localization data.
- **Lightweight real-time localization**: implements a simplified localization system using AprilTags and watchtower cameras, optimizing computational efficiency for real-time tracking.
- **Decentralized rescue architecture**: employs a central Rescue Center and multiple Rescue Agents, each dedicated to an individual Duckiebot, enabling simultaneous rescues.
- **Closed-loop control for recovery**: uses a proportional-integral (PI) controller to execute corrective movements, bringing Duckiebots back to lane-following mode.
City Rescue is a great example of vehicle-to-infrastructure (v2i) interactions in Duckietown.
[ Learn robot autonomy with Duckietown ](https://duckietown.com/educational-resources/#autonomy)
## The challenges and approach
The City Rescue autonomous recovery system employs a server-based architecture, where a central “Rescue Center” continuously processes localization data and assigns rescue tasks to dedicated Rescue Agents.
The localization system uses appropriately placed reference AprilTags and watchtower cameras, tuned for low-latency operation by bypassing computationally expensive optimization routines. The rescue mechanism is driven by a [PI controller](https://duckietown.com/educational-resources/#education-materials-modcon "Modeling and Control Duckietown pedagogical resources"), which calculates corrective movements based on deviations from an ideal trajectory.
The main challenges in implementing this city behavior include localization inaccuracies, due to the limited coverage of watchtower cameras, and distress event positioning on the map.
The localization inaccuracies are mitigated by performing camera calibration procedures on the watchtower cameras, as well as by performing an initial city offset calibration procedure. The success rate of the executed maneuvers varies with map topographical complexity; recovery from curved road or intersection sections is less reliable than from straight lanes.
Finally, the lack of inter-robot communication can lead to cascading failure scenarios when multiple Duckiebots collide.
## City rescue: full report
The design and implementation of this autonomous recovery system is documented in the following report.
## City rescue in Duckietown: Authors
[  ](https://www.linkedin.com/in/carl-philipp-biagosch/)
[Carl Philipp Biagosch](https://www.linkedin.com/in/carl-philipp-biagosch/) is the co-founder at [Mantis Ropeway Technologies](https://www.linkedin.com/company/mantisropewaytechnologies/), Switzerland.
[  ](https://www.linkedin.com/in/jasonhsj/)
[Jason Hu](https://www.linkedin.com/in/jasonhsj/) is currently working as a Scientific Assistant at [ETH Zurich](https://ethz.ch/en.html), Switzerland.
[  ](https://www.linkedin.com/in/martin-xu-341931145/)
[Martin Xu](https://www.linkedin.com/in/martin-xu-341931145/) is currently working as a data scientist at [QuantCo](http://www.quantco.com/), Germany.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Projects
**Tags:** project, student project
---
### [University of Nevada doing IoT with Duckietown](https://duckietown.com/university-of-nevada-doing-iot-with-duckietown/)
**Published:** January 17, 2023
**Author:** Ivano Marocchi
**Content:**
## University of Nevada doing IoT with Duckietown
Here’s an extract from Nevada Today’s article “Integrating big data into robotics with Duckietown”, written by Kaeli Britt.
[ Read the full article here ](https://www.unr.edu/nevada-today/news/2022/duckietown)
For the third year, the University of Nevada, Reno’s Computer Science & Engineering (CSE) department conducted a Research Experience for Teachers (RET) program focused on “Integrating Big Data into Robotics.”
Through the six-week program, participants were able to gain hands-on robotics experience that can be applied in classrooms later, in a fun, nontraditional way.
Duckietown, an engineering and robotics/artificial intelligence (AI) project, focuses on accessible and engaging styles of learning. The project started at the Massachusetts Institute of Technology in 2016 as a graduate class, where they created a video “Duckumentary” highlighting the background and purpose of the research project but also its adaptability for varying age groups.
This year’s University project was taught by Ph. D. candidate and instructor Amirhesam Yazdi as well as CSE associate professor and principal investigator Lei Yang.
Participants were able to learn how to assemble the robots, build and design the track, and program the robots and the track.
“Duckietown is a freely available robotics platform and curricula for all levels of education. It is tangible, accessible, and fun. It has mobile robots and roads, constructed from exercise mats and tape,” Yang said. “The mobile robots are built from off-the-shelf parts and using open-source software and the curricula, such as lectures and exercises are provided on the Duckietown website. These unique features set Duckietown apart from other engineering, robotics and/or AI projects.”
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** dep-News, News, outreach
---
### [There's MAGIC in Westminster: innovating and cultivating, an interview with Graham Dodge](https://duckietown.com/theres-magic-in-westminster-innovating-and-cultivating-an-interview-with-graham-dodge/)
**Published:** September 4, 2023
**Author:** Federico Tani
**Content:**
**Westminster, Maryland**: MAGIC and Duckietown partner to bring robot autonomy education in Westminster, MD.
The **Mid-Atlantic Gigabit Innovation Collaboratory** (MAGIC) was awarded funding from the Knorr-Bremse Global Care program in July of 2022 to launch an Autonomous Robotics Innovation Center (ARIC) in the heart of downtown Westminster.
Their goal for ARIC is to use the Duckietown platform to give Westminster students industry-relevant skills and hands-on experience that will prepare them for careers or further learning in robotics and engineering.
##### Quick links
- [MAGIC website](https://magicinc.org/)
- [Duckietown in Wesminster](https://magicinc.org/duckietown-in-westminster)
- [ARIC](https://aricmd.org/)
## Bringing industry relevant skills and hands-on experience to high school and undergrad students
We are very pleased to announce our partnership with MAGIC, bringing the Duckietown platform to Carroll County students and introducing the local community to robotics and AI technology.
MAGIC is a 501(c)3 non-profit organization headquartered in Westminster, MD, USA. Their mission is to build a technological ecosystem that creates and nurtures talent, entrepreneurship, and tech businesses, elevating the Westminster Gigabit community to lead the Mid-Atlantic region.
We talked with Graham Dodge, Executive Director of MAGIC, to know more about their new Autonomous Robotics Innovation Center (ARIC) project and how Duckietown is being used in this context.
###### Hi! Could you introduce yourself?
My name is Graham Dodge. I’m the executive director of Magic Inc., a 501(c)3 non-profit based in Maryland, United States. We focus on technology, education, and workforce development within our community.
###### Thank you for taking the time for this interview, Graham. How did you learn about Duckietown?
I was speaking with the CEO of a local company called Dynamic Dimension Technologies. We were discussing our autonomous corridor project in the city of Westminster, Maryland. He recommended using Duckietown as a platform to demonstrate some of the technologies to local elected officials and stakeholders. After seeing videos of Duckietown on their website, I was impressed by its user-friendly and cute approach to making technology accessible to people unfamiliar with computer vision and robotics. It seemed like a perfect way to explain these concepts to our community.
###### Very interesting! What are you using Duckietown for?
Initially, we used Duckietown to demonstrate autonomous technologies for our corridor project. However, we later secured funding and set up the Autonomous Robotics Innovation Center (ARIC), where we now use Duckietown for a broader robotics program. In ARIC, we teach students ROS, Python, Linux, and the documentation that comes along with Duckietown. Additionally, we plan to integrate railroad intersections with Duckietown, thanks to grant funding from Knorr-Bremse Global Care program in Germany, a train parts manufacturer involved in autonomous train systems.
> “We’re seeing that students with very minimal computer science education can jump right into Duckietown and excel”.
>
> Graham Dodge
###### That’s impressive, so you are integrating Duckietown with a rail system?
Yes, we are considering names like Duckierail or Duckietrains. This project aims to develop open-source solutions for Duckiebots to interact with trains and railroad crossings. We are excited to collaborate with the Knorr-Bremse Global Care program, who provided the funding, to showcase rail systems in a smart, connected infrastructure with autonomous vehicles.
###### We can’t wait to see the developments! Are you satisfied with your experience with Duckietown so far?
Absolutely! We started with a small MOOC kit, and despite lacking an internal instructor, two student interns managed to get the Duckiebots up and running within three months. The hands-on experience has been rewarding for them, and it’s remarkable how much they’ve learned. We are now expanding the program and have plans to engage with local colleges as well.
###### That’s fantastic! So, do you see Duckietown as a valuable tool for education and career development in the robotics industry?
Yes, absolutely! Our goal is to set up an apprenticeship program for career pathway development. By providing students with hands-on experience through Duckietown, we aim to make them more attractive to local employers in the robotics industry. The students are already gaining valuable skills that match or even exceed those of some graduate students seeking internships with the same employers.
> “We’ve had industry professionals and
> local employers look at what we’re teaching and say that what our
> high school interns are learning and building with Duckietown is more advanced – and
> more industry relevant – than most of the undergraduate students they’ve seen”.
>
> Graham Dodge
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[Get Started with Duckietown!](https://www.duckietown.com/guides)
[Find out more use cases](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
**Categories:** Blog, News, People
---
### [Implementing vision based dynamic obstacle avoidance](https://duckietown.com/dynamic-obstacle-avoidance-in-duckietown/)
**Published:** July 6, 2024
**Author:** Duckietown Admin
**Excerpt:** This student project implements dynamic obstacle avoidance for Duckiebots with the aim of detecting and navigating around static and moving obstacles.
**Content:**
# Implementing vision based dynamic obstacle avoidance
##### Project Resources
- **Objective**: The objective of this project was to implement dynamic obstacle avoidance for Duckiebots.
- **Approach**: Enhancing lane-following algorithms to detect and navigate around static and moving obstacles.
- **Authors**: Nikolaj Witting, Fidel Esquivel Estay, Johannes Lienhart, and Paula Wulkop
[ Final Result ](#project-result)
[ Code ](https://github.com/duckietown-ethz/proj-lfvop)
[ Authors ](#authors)
## Why dynamic obstacle avoidance?
Dynamic obstacle avoidance is the process of detecting a region of space that is not navigable (an obstacle), planning a path around it, and executing that plan.
When the obstacle moves, the plan needs to account for the future positions of the object as well, making the process significantly more complicated than passing a static obstacle.
With this aim, the authors of this project designed and implemented a robust passing algorithm for Duckiebots in Duckietown.
The approach adopted was to develop a new LED-based detection system, modify the typical Duckietown lane following pipeline for planning around the obstacles, and deploying a new controller to execute manoeuvres.
[ Learn computer vision with Duckietown ](https://duckietown.com/educational-resources/#education-materials-cv)
## Dynamic obstacle avoidance:
the challenges
Some of the key challenges associated with this project are the following:
**Detection Accuracy:** The Duckiebot and Duckies detection systems occasionally produce false positives. Light sources from other Duckiebots or shiny objects can interfere with the LED detection, while yellow line segments can be mistaken for Duckies. Improving the reliability of detection under varying lighting conditions is essential.
**Lane Following Stability:** The Duckiebots sometimes become unstable while overtaking, especially when driving in the left lane. The lane-following system struggles with large lane pose angles or rapid changes in lane position, which can cause the Duckiebot to veer off the road. Enhancing the lane-following algorithm to maintain stability during lane changes is critical.
**Velocity Estimation:** Estimating the speed of moving Duckiebots accurately is challenging. The current position data obtained from LED detection fluctuates too much to provide a reliable velocity measurement. Developing a more robust method for estimating the velocity of other Duckiebots is needed to ensure safe and efficient overtaking.
**Variable Speed Control:** Implementing variable speed control during overtaking is problematic due to instability in the lane-following pipeline when speeds are dynamically adjusted. Adjusting speed based on the detected obstacle’s speed without losing lane stability is difficult, necessitating improvements in the lane control model to handle speed changes effectively.
## Project Highlights
Here is the output of their work. [Check out the github repository ](#links "project resources")for more details!
![For Dynamic Obstacle Avoidance, as shown on the left a slowly moving or static Duckiebot is overtaken, while on the right the rear Duckiebot is staying behind the obstacle to avoid oncoming traffic.]()
Fig. 1. Illustration of the mission.
![The cropped input image seen on top, with the thresholded binary image below.]()
Fig. 2. Cropped Input and Thresholded Binary Image Comparison.
![The similarity between the red and white LEDs as percieved by the Duckiebot is illustrated here. On the left are the red LED and on the right is the white LED. In the bottom half, the center color of each LED is drawn through itself and the opposite LED to show the similarity between them.]()
Fig. 3. The similarity between the red and white LEDs as percieved by the Duckiebot.
![Yellow mask of an image with Duckie on lane.]()
Fig. 4. Yellow mask of an image with Duckie on lane.
![Duckiebot driving towards Duckie]()
Fig. 5. Duckiebot driving towards Duckie.
![This image is a flowchart for the demonstration of the decision logic for this project.]()
Fig. 6. Decision logic for this project.
![Lane offset response to linearly changed d_offset parameter. The step response of the d_offset for a single overtaking action was tested.]()
Fig. 7. Lane offset response to linearly changed d\_offset parameter.
![This image shows the Overtaking maneuver of the Duckiebot.]()
Fig. 8. Overtaking maneuver.
## Dynamic Obstacle Avoidance: Results
## Dynamic Obstacle Avoidance: Authors
[  ](https://www.linkedin.com/in/nikolaj-witting-a2395a131/)
[Nikolaj Witting](https://www.linkedin.com/in/nikolaj-witting-a2395a131/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works at [Trackman](https://www.trackman.com/) as an Algorithm Developer.
[  ](https://www.linkedin.com/in/fidel-esquivel/)
[Fidel Esquivel Estay](https://www.linkedin.com/in/fidel-esquivel/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), currently serving as the Co-Founder at [UpCircle](https://www.upcircle.ai/).
[  ](https://www.linkedin.com/in/johannes-lienhart-906143164/)
[Johannes Lienhart](https://www.linkedin.com/in/johannes-lienhart-906143164/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), currently serving as the CTO at [Tethys Robotics](https://www.tethys-robotics.ch/).
[  ](https://www.linkedin.com/in/paula-wulkop/)
[Paula Wulkop](https://www.linkedin.com/in/paula-wulkop/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), where she is currently pursuing her Ph.D.
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Projects
**Tags:** project, student project
---
### [Proxy Domains for Evaluation and Learning](https://duckietown.com/proxy-domains-for-evaluation-and-learning-in-duckietown/)
**Published:** March 15, 2025
**Author:** Duckietown Admin
**Excerpt:** This research defines metrics to quantify proxy domains' in robotics, using PRPV and PLV to assess transfer learning, and domain adaptation in Duckietown.
**Content:**
##### General Information
- **Title**: On Assessing the Usefulness of Proxy Domains for Developing and Evaluating Embodied Agents
- **Authors**: Anthony Courchesne, Andrea Censi, Liam Paull
- **Institution**: Montreal Robotics and Embodied AI Lab (REAL) at Université de Montréal, Mila.
- **Citation**: A. Courchesne, A. Censi and L. Paull, "On Assessing the Usefulness of Proxy Domains for Developing and Evaluating Embodied Agents," 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 2021, pp. 4298-4305, doi: 10.1109/IROS51168.2021.9635977.
[ Paper ](https://ieeexplore.ieee.org/document/9635977)
[ Authors ](#authors)
[ Institution ](https://montrealrobotics.ca/)
# Proxy Domains for Evaluation and Learning
Running robotics experiments in the real world is often costly in terms of time, money, and effort. For this reason, robotics development and testing rely on **proxy domains** (e.g., simulations) before real-world deployment. But how to gauge the degree of usefulness of using proxy domains in the development process, and are all domains equally useful?
Intuitively, the answer to the above questions will depend on the type of robot, the task it has to achieve, and the environment in which it operates. Evaluating a proxy domain’s usefulness for a specific combination of these circumstances, specifically for the training of autonomous agents, is tackled in this work by establishing quantification metrics and assessing them in Duckietown.
The key aspects of this work are:
- **Proxy Usefulness Metrics**: introduction of **Proxy Relative Predictivity Value** (PRPV) and **Proxy Learning Value** (PLV) to measure a proxy’s ability to predict real-world performance and aid agent learning. PRPV helps identify simulations that accurately predict real-world results, while PLV measures their effectiveness in training agents.
- **Prediction vs. Learning**: differentiation of proxies used for accurate performance prediction from those for data generation in training.
- **Experiments**: demonstration of how tuning proxy domain parameters (e.g., sensor delays, camera angle) affects predictivity and learning efficiency.
These metrics improve proxy selection and tuning for robotics research and education, and Duckietown enables rapid prototyping of these ideas for mobile autonomous vehicles.
[ Learn about machine learning with Duckietown materials ](https://duckietown.com/educational-resources/)
## Highlights - Proxy Domains for Evaluation and Learning in Duckietown
Here is a visual tour of the work of the authors. For all the details, check out the [full paper](#links "Post resources").
![Proxy domains and Duckietown comparison for evaluating robotics task performance.]()
Figure 1. Comparing Proxy Domain Performance in Robotics
![Proxy domains and Duckietown agent interaction for simulation vs. real robot evaluation.]()
Figure 2. Agent-Environment Interface in Proxy and Target Domains
![Proxy domains and Duckietown usefulness metrics including PRPV, PPV, SRCC, and POD.]()
Figure 3. Proxy Usefulness Metrics Comparison
![Proxy domains and Duckietown evaluation for AI Driving Olympics parameter optimization.]()
Figure 4. Evaluating Duckietown Proxy Domains for AIDO Challenge
![Proxy domains and Duckietown side-by-side visualization of environment and trajectories.]()
Figure 5. Proxy vs. Target Domain in Duckietown
![Proxy domains and Duckietown performance comparison for imitation learning agents.]()
Figure 6. Performance of an Imitation Learning Agent in Duckietown
[](https://duckietown.com/wp-content/uploads/2025/03/Figure-1.-Comparing-Proxy-Domain-Performance-in-Robotics.avif) Figure 1. Comparing Proxy Domain Performance in Robotics [](https://duckietown.com/wp-content/uploads/2025/03/Figure-2.-Agent-Environment-Interface-in-Proxy-and-Target-Domains-png.avif) Figure 2. Agent-Environment Interface in Proxy and Target Domains [](https://duckietown.com/wp-content/uploads/2025/03/Figure-3.-Proxy-Usefulness-Metrics-Comparison-png.avif) Figure 3. Proxy Usefulness Metrics Comparison
[](https://duckietown.com/wp-content/uploads/2025/03/Figure-4.-Evaluating-Duckietown-Proxy-Domains-for-AIDO-Challenge-png.avif) Figure 4. Evaluating Duckietown Proxy Domains for AIDO Challenge [](https://duckietown.com/wp-content/uploads/2025/03/Figure-5.-Proxy-vs.-Target-Domain-in-Duckietown.avif) Figure 5. Proxy vs. Target Domain in Duckietown [](https://duckietown.com/wp-content/uploads/2025/03/Figure-6.-Performance-of-an-Imitation-Learning-Agent-in-Duckietown-png.avif) Figure 6. Performance of an Imitation Learning Agent in Duckietown
## Abstract
In the author’s words:
In many situations it is either impossible or impractical to develop and evaluate agents entirely on the target domain on which they will be deployed. This is particularly true in robotics, where doing experiments on hardware is much more arduous than in simulation. This has become arguably more so in the case of learning-based agents. To this end, considerable recent effort has been devoted to developing increasingly realistic and higher fidelity simulators. However, we lack any principled way to evaluate how good a “proxy domain” is, specifically in terms of how useful it is in helping us achieve our end objective of building an agent that performs well in the target domain. In this work, we investigate methods to address this need. We begin by clearly separating two uses of proxy domains that are often conflated: 1) their ability to be a faithful predictor of agent performance and 2) their ability to be a useful tool for learning. In this paper, we attempt to clarify the role of proxy domains and establish new proxy usefulness (PU) metrics to compare the usefulness of different proxy domains. We propose the relative predictive PU to assess the predictive ability of a proxy domain and the learning PU to quantify the usefulness of a proxy as a tool to generate learning data. Furthermore, we argue that the value of a proxy is conditioned on the task that it is being used to help solve. We demonstrate how these new metrics can be used to optimize parameters of the proxy domain for which obtaining ground truth via system identification is not trivial.
## Conclusion - Proxy Domains for Evaluation and Learning in Duckietown
Here are the conclusions from the author of this paper:
“We introduce new metrics to assess the usefulness of proxy domains for agent learning. In a robotics setting it is common to use simulators for development and evaluation to reduce the need to deploy on real hardware. We argue that it is necessary to to take into account the specific task when evaluating the usefulness of the the proxy. We establish novel metrics for two specific uses of a proxy. When the proxy domain is used to predict performance in the target domain, we offer the PRPV to assess the usefulness of the proxy as a predictor, and we argue that the task needs to be imposed but not the agent. When a proxy is used to generate training data for a learning algorithm, we propose the PLV as a metric to assess usefulness of the source domain, which is dependent on a specific task and a learning algorithm. We demonstrated the use of these measures for predicting parameters in the Duckietown environment. Future work will involve more rigorous treatment of the optimization problems posed to find optimal parameters, possibly in connection with differentiable simulation environments.”
#### Project Authors
[  ](https://www.linkedin.com/in/courchesnea/?originalSubdomain=ca)
[Anthony Courchesne](https://www.linkedin.com/in/courchesnea/?originalSubdomain=ca) is currently working as an MLOps Engineer ar [Maneva](https://www.maneva.ai/), Canada.
[  ](https://www.linkedin.com/in/censi/)
[Andrea Censi](https://www.linkedin.com/in/censi/) is currently working as the Deputy Director, Chair of Dynamic Systems and Control at [ETH Zurich](http://www.ethz.ch/), Switzerland.
[  ](https://www.linkedin.com/in/liam-paull-83a5442b/?originalSubdomain=ca)
[Liam Paull](https://www.linkedin.com/in/liam-paull-83a5442b/?originalSubdomain=ca) is an Associate Professor at the [Universite de Montreal](http://www.umontreal.ca/), Canada and also serves as the Chief Education Officer at [Duckietown](https://duckietown.com/).
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** research
---
### [Adaptive Lane Following with Auto-Trim Tuning](https://duckietown.com/adaptive-lane-following-with-automatic-trim-calibration/)
**Published:** March 7, 2025
**Author:** Duckietown Admin
**Excerpt:** This project enhances Adaptive Lane Following by enabling Duckiebots to autonomously calibrate wheel trim, ensuring stable navigation without manual tuning.
**Content:**
# Adaptive Lane Following with Auto-Trim Tuning
##### Project Resources
- **Objective**: Develop an Adaptive Controller for Duckiebots to autonomously calibrate wheel trim, ensuring stable lane following without manual tuning.
- **Approach**: Implement a Model-Reference Adaptive Control (MRAC) system that continuously adjusts trim based on real-time lane position feedback.
- **Authors**: Pietro Griffa, Simone Arreghini, Rohit Suri, Aleksandar Petrov, Jacopo Tani
[ Report ](#thesis)
[ University ](https://ethz.ch/en.html)
[ Authors ](#authors)
[ Code ](https://github.com/duckietown-ethz/proj-lf-adaptive)
#### Before and after:
#### Training:
## Project highlights
Calibration of sensor and actuators is always important in setting up robot systems, especially in the context of autonomous operations. Manual tweaking of calibration parameters though is a nuisance, albeit necessary when every physical instance of the robots is slightly different from each other.
In this project, the authors developed a process to automatically calibrate the trim parameter in the Duckiebot, i.e., allowing it to go straight when an equal command to both wheel motors is provided.
## Adaptive lane following in Duckietown: beyond manual odometry calibration
The objective of this project is to develop a process to autonomously calibrate the wheel trim parameter of Duckiebots, eliminating the need for manual tuning or improving upon it. Manual tuning of this parameter, as part of the odometry calibration procedure, is needed to account for the invevitable slight differences existing across different Duckiebots, due to manufacturing, assembly, handling difference, etc.
Creating an automatic trim calibration procedure enhances the Duckiebot’s lane following behavior, by continuously adjusting the wheel alignment based on real-time lane pose feedback. Duckiebots typically require manual calibration for the odometry, which introduces variability and reduces scalability in autonomous mobility experiments.
By implementing a [Model-Reference Adaptive Control](https://www.mathworks.com/help/slcontrol/ug/model-reference-adaptive-control.html) (MRAC) based approach, the project ensures consistent performance despite mechanical variations or external disturbances. This is desireable for large-scale Duckietown deployments where the robots need to maintain uniform behavior across different assemblies.
Adaptive control reduces dependence on predefined parameters, allowing Duckiebots to self-correct without external intervention. This enables more reproducible fleet-level performance, useful for research in autonomous navigation. This project supports experimentation in self-calibrating robotic systems through application of adaptive control research.
[ Learn robot autonomy with Duckietown ](https://duckietown.com/educational-resources/#autonomy)
## Model Reference Adaptive Control (MRAC) for adaptive lane following in Duckietown
The method employs a Model-Reference Adaptive Control (MRAC) framework that iteratively estimates the optimal trim value during lane following by processing lane pose feedback from the vision pipeline, and comparing expected and actual motion to compute a correction factor. An adaptation law updates the trim dynamically based on real-time error minimization.
Pose estimation relies on a vision-based lane filter, which introduces latency and noise, affecting convergence stability. The adaptive controller must maintain stability while ensuring convergence to an optimal trim value within a finite time window.
The performance of this approach is constrained by sensor inaccuracies, requiring threshold-based filtering to exclude unreliable pose data. The algorithm operates in real-world conditions where road surface variations, lighting changes, and mechanical wear affect performance. Synchronizing lane pose data with controller updates while minimizing computation delays is a key challenge, and ensuring that the adaptive controller does not introduce oscillations or instability in the control loop requires parameter tuning.
## Adaptive lane following: full report
Check out the full report here.
## Adaptive lane following in Duckietown: Authors
[  ](https://www.linkedin.com/in/pietrogriffa/?locale=en_US)
[Pietro Griffa](https://www.linkedin.com/in/pietrogriffa/?locale=en_US) is currently working as a Systems and Estimation Engineer at [Verity](https://www.verity.net/), Switzerland.
[  ](https://www.linkedin.com/in/simone-arreghini/?originalSubdomain=ch)
[Simone Arreghini](https://www.linkedin.com/in/simone-arreghini/?originalSubdomain=ch) is currently pursuing his Ph. D. at [IDSIA USI-SUPSI](http://www.idsia.ch/), Switzerland.
[  ](https://www.linkedin.com/in/suri-rohit/?originalSubdomain=sg)
[Rohit Suri](https://www.linkedin.com/in/suri-rohit/?originalSubdomain=sg) was a mentor on this project and is currently working as a Senior Research Scientist at [Venti Technologies](https://www.ventitech.ai/), Singapore.
[  ](https://www.linkedin.com/in/aleksandar-petrov/?originalSubdomain=uk)
[Aleksandar Petrov](https://www.linkedin.com/in/aleksandar-petrov/?originalSubdomain=uk) was a mentor on this project and is currently pursuing his Ph. D. at the [University of Oxford](http://www.ox.ac.uk/), United Kingdom.
[  ](https://www.linkedin.com/in/aleksandar-petrov/?originalSubdomain=uk)
[Jacopo Tani](https://www.linkedin.com/in/jacopo-tani/) was a supervisor on this project and is currently the CEO at Duckietown.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Projects
**Tags:** project, student project
---
### [Deep Reinforcement and Transfer Learning for Robot Autonomy](https://duckietown.com/deep-reinforcement-and-transfer-learning-for-robot-autonomy/)
**Published:** February 22, 2025
**Author:** Duckietown Admin
**Excerpt:** This work explores deep reinforcement learning (DRL) agents trained in simulation and transferred using Duckietown for real-world validation.
**Content:**
##### General Information
- **Title**: Agent-Based Autonomous Robotic System Using Deep Reinforcement and Transfer Learning
- **Authors**: Vladyslav Kyryk, Maksym Figat, Marian Kyryk
- **Institution**: Warsaw University of Technology, Warsaw, Poland
- **Citation**: Kyryk, V., Figat, M., Kyryk, M. (2024). Agent-Based Autonomous Robotic System Using Deep Reinforcement and Transfer Learning. In: Luntovskyy, A., Klymash, M., Melnyk, I., Beshley, M., Schill, A. (eds) Digital Ecosystems: Interconnecting Advanced Networks with AI Applications. TCSET 2024. Lecture Notes in Electrical Engineering, vol 1198. Springer, Cham.
[ Paper ](https://link.springer.com/chapter/10.1007/978-3-031-61221-3_23)
[ Authors ](#authors)
[ Institution ](https://eng.pw.edu.pl/)
# Deep Reinforcement and Transfer Learning for Robot Autonomy
Developing autonomous robotic systems is challenging. When using machine learning based approaches, one of the main challenges is the **high cost** **and complexity of real-world training**. Running real world experiments is time consuming and depending on the application, can be expensive as well.
This work uses **Deep Reinforcement Learning (DRL)** and tackles this challenge through **Transfer Learning (TL)**. DRL enables robots to learn optimal behaviors through trial-and-error, guided by reward-based feedback. Transfer Learning then addresses the high cost of generating training data by leveraging simulation environments.
Running experiments in simulation is time and cost efficient, the trained agent can then be deployed on a physical robot, in a process known as **Sim2Real** transfer. Ideally, this approach significantly reduces training costs and accelerates real-world deployment.
In this work, training occurs in a simulated Duckietown environment using **Deep Deterministic Policy Gradient (DDPG)** and TL techniques to mitigate the expected difference between simulated and real-world environments. The resulting agent is then deployed on a custom-built robot in a physical Duckietown city for evaluation.
Results show that the DRL-based model **successfully learns** lane-following and navigation autonomous behaviors in simulation, and performance comparison with real world experiments is provided.
[ Learn about machine learning with Duckietown materials ](https://duckietown.com/educational-resources/)
## Highlights - Deep Reinforcement Learning for Agent-Based Autonomous Robot
Here is a visual tour of the work of the authors. For all the details, check out the [full paper](#links "Post resources").
![Diagram showing agent-environment interaction in deep reinforcement learning, applied to Duckietown for autonomous robot training.]()
Figure 1. Agent-Environment Interaction in Reinforcement Learning.
![Chart categorizing reinforcement learning algorithms used in deep reinforcement learning for Duckietown experiments.]()
Figure 2. Classification of RL Algorithms.
![Diagram showing the actor-critic method in deep reinforcement learning, applied to Duckietown robots.]()
Figure 3. Actor-Critic Method Architecture.
![Diagram specifying different computational agents in a deep reinforcement learning system for Duckietown.]()
Figure 4. Specification of Computational Agents.
![Structural diagram of an embodied agent used in deep reinforcement learning for Duckietown robots.]()
Figure 5. General Embodied Agent Structure.
![Finite-state machine (FSM) of the coordinator agent in a deep reinforcement learning system for Duckietown.]()
Figure 6. FSM of Coordinator Agent.
![System architecture showing all agents created and the coordinator agent running in a deep reinforcement learning Duckietown setup.]()
Figure 7. System Structure with Active Coordinator Agent.
![Structural design of a robot agent used in deep reinforcement learning experiments in Duckietown.]()
Figure 8. Embodied Agent Structure of Robot Agent.
![Finite-state machine (FSM) representation of a robot agent in deep reinforcement learning experiments in Duckietown.]()
Figure 9. FSM of Robot Agent.
![Extended finite-state machine (FSM) of the wrapper agent in a deep reinforcement learning system for Duckietown.]()
Figure 10. Extended FSM of Wrapper Agent.
![Comparison of ARS implementation in real and simulated environments for deep reinforcement learning in Duckietown.]()
Figure 11. Implementation of ARS in Real and Simulated Environments.
![Finite-state machine (FSM) of the image wrapper in a deep reinforcement learning system for Duckietown.]()
Figure 12. Image Wrapper FSM.
![A simple loop track built for testing deep reinforcement learning in Duckietown, comparing simulator and real-world environments.]()
Figure 13. Simple Loop Built as Part of Real-World ARS.
![Robot trajectories on evaluation maps in a deep reinforcement learning simulator for Duckietown, with a green dot marking the starting point.]()
Figure 14. Robot Trajectories on Evaluation Maps in Simulator.
![Side-by-side comparison of raw observations from simulated and real-world environments in Duckietown deep reinforcement learning experiments.]()
Figure 13. Raw Observations from Both Environments.
[](https://duckietown.com/wp-content/uploads/2025/02/Agent-Environment-Interaction-in-Reinforcement-Learning-png.avif) Figure 1. Agent-Environment Interaction in Reinforcement Learning. [](https://duckietown.com/wp-content/uploads/2025/02/Classification-of-RL-Algorithms.avif) Figure 2. Classification of RL Algorithms. [](https://duckietown.com/wp-content/uploads/2025/02/Actor-Critic-Method-Architecture.avif) Figure 3. Actor-Critic Method Architecture.
[](https://duckietown.com/wp-content/uploads/2025/02/Specification-of-Computational-Agents-png.avif) Figure 4. Specification of Computational Agents. [](https://duckietown.com/wp-content/uploads/2025/02/General-Embodied-Agent-Structure.avif) Figure 5. General Embodied Agent Structure. [](https://duckietown.com/wp-content/uploads/2025/02/FSM-of-Coordinator-Agent-png.avif) Figure 6. FSM of Coordinator Agent.
[](https://duckietown.com/wp-content/uploads/2025/02/System-Structure-with-Active-Coordinator-Agent-png.avif) Figure 7. System Structure with Active Coordinator Agent. [](https://duckietown.com/wp-content/uploads/2025/02/Embodied-Agent-Structure-of-Robot-Agent-png.avif) Figure 8. Embodied Agent Structure of Robot Agent. [](https://duckietown.com/wp-content/uploads/2025/02/FSM-of-Robot-Agent.avif) Figure 9. FSM of Robot Agent.
[](https://duckietown.com/wp-content/uploads/2025/02/Extended-FSM-of-Wrapper-Agent-png.avif) Figure 10. Extended FSM of Wrapper Agent. [](https://duckietown.com/wp-content/uploads/2025/02/Implementation-of-ARS-in-Real-and-Simulated-Environments.avif) Figure 11. Implementation of ARS in Real and Simulated Environments. [](https://duckietown.com/wp-content/uploads/2025/02/Image-Wrapper-FSM-png.avif) Figure 12. Image Wrapper FSM.
[](https://duckietown.com/wp-content/uploads/2025/02/Simple-Loop-Built-as-Part-of-Real-World-ARS.avif) Figure 13. Simple Loop Built as Part of Real-World ARS. [](https://duckietown.com/wp-content/uploads/2025/02/Raw-Observations-from-Both-Environments.avif) Figure 13. Raw Observations from Both Environments. [](https://duckietown.com/wp-content/uploads/2025/02/Robot-Trajectories-on-Evaluation-Maps-in-Simulator.avif) Figure 14. Robot Trajectories on Evaluation Maps in Simulator.
## Abstract
In the author’s words:
Real robots have different constraints, such as battery capacity limit, hardware cost, etc., which make it harder to train models and conduct experiments on physical robots. Transfer learning can be used to omit those constraints by training a self-driving system in a simulated environment, with a goal of running it later in a real world. Simulated environment should resemble a real one as much as possible to enhance transfer process. This paper proposes a specification of an autonomous robotic system using agent-based approach. It is modular and consists of various types of components (agents), which vary in functionality and purpose.
Thanks to system’s general structure, it may be transferred to other environments with minimal adjustments to agents’ modules. The autonomous robotic system is implemented and trained in simulation and then transferred to real robot and evaluated on a model of a city. A two-wheeled robot uses a single camera to get observations of the environment in which it is operates. Those images are then processed and given as an input to the deep neural network, that predicts appropriate action in the current state. Additionally, the simulator provides a reward for each action, which is used by the reinforcement learning algorithm to optimize weights in the neural network, in order to improve overall performance.
## Conclusion - Deep Reinforcement Learning for Agent-Based Autonomous Robot
Here are the conclusions from the author of this paper:
“After several breakthroughs in the field of Deep Reinforcement Learning, it became one of the most popular researched topics in Machine Learning and a common approach to the problem of autonomous driving. This paper presents the process of training an autonomous robotic system using popular actor-critic algorithm in the simulator, which may then also be run on real robot. It was possible to train an agent in real-time using trial-and-error approach without the need to collect vast amounts of labeled data. The neural network learned how to control the robot and how to follow the lanes, without any explicit guidelines. Only a few functions have been used to transform the data sent between environment and the agent, in order to make the learning process smoother and faster.
For evaluation purposes, a real robot and a small city model have been built, based on the [Duckietown](/) platform specification. This hardware has been used to evaluate in the real world the performance of the system, trained in simulator. Also, additional Transfer Learning techniques were used, in order to adjust the observations and actions in the real robot, due to the differences with simulated environment. Although, the performance in real environment was worse than in simulator, certain trained models were still able to guide the robot around a simple road loop, which shows a potential for such approach. As a result, the use of the simulator greatly reduced the time and effort needed to train the system, and transfer methods were used to deploy it in the real world.
The Duckietown platform provides a baseline, which was modified and refactored to follow the system structure. The simulator and its components are thoroughly documented, the detailed instructions explain how to train and run the robot both in simulation and in real world and evaluate the results. Duckietown provides complete sets of parts, necessary to build the robot and small city, however, it was decided to build custom robot, according to the guidelines. The robot uses a single camera to get observations of the surrounding environment.
The reinforcement learning algorithm was used to learn a policy, which tries to choose optimal actions based on the those observations with the help of reward function, that provides a feedback for previous decisions. It was possible to significantly reduce the effort required to train a model, thanks to the simulator, as the process does not require constant human supervision and involvement. Such approach proves to be very promising, as the agent learned how to do the lane-following task without any explicit labels, and has shown good performance in the simulated environment. Although, there is still a room for improvement, when it comes to transferring the model to real world, which requires various adaptations and adjustments to be made for The robot to properly execute maneuvers and show stability in its actions.”
#### Project Authors
[  ](https://www.linkedin.com/in/vladyslav-kyryk-96483a2a4/?originalSubdomain=pl)
[Vladyslav Kyryk](https://www.linkedin.com/in/vladyslav-kyryk-96483a2a4/?originalSubdomain=pl) is currently working as a Data Scientist at [Finitec](https://www.finitec.pl/pl/), Warsaw, Poland.
[  ](https://www.linkedin.com/in/maksym-figat/?originalSubdomain=pl)
[Maksym Figat](https://www.linkedin.com/in/maksym-figat/?originalSubdomain=pl) is working as an Assistant Professor at [Warsaw University of Technology](https://www.pw.edu.pl/), Poland.
[  ](https://www.linkedin.com/in/maryan-kyryk-/?originalSubdomain=ua)
[Maryan Kyryk](https://www.linkedin.com/in/maryan-kyryk-/?originalSubdomain=ua) is currently serving as the Co-Founder & CEO at [Maxitech](https://www.maxitech.com.ua/), Ukraine.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** research
---
### [Flexible tether control in marsupial systems](https://duckietown.com/flexible-tether-control-in-heterogeneous-marsupial-systems/)
**Published:** February 15, 2025
**Author:** Duckietown Admin
**Excerpt:** This project develops a flexible tether control system for Duckiebot, using ROS and an automated spool to optimize tether length for mobility and efficiency.
**Content:**
# Flexible tether control in marsupial systems
##### Project Resources
- **Objective**: Develop a control system to optimize flexible tethered robotic systems with Duckiebot passenger vehicles.
- **Approach**: Using ROS, and Duckiebot (DB21J) sensor suite to dynamically control tether slackness and tension.
- **Authors**: Carson Duffy, Dr. Jason O’Kane
[ Report ](#thesis)
[ University ](https://www.tamu.edu/index.html)
[ Authors ](#authors)
## Project highlights
Wouldn’t it be great to have a base station transfer power, data and other information to other autonomous vehicles through a tethered connection? But how to deal with the challenges arising from controlling the length and tension of the tether?
Here is an overview of the authors’ results:
![Isometric view of a CAD model of an automated spool designed for flexible tether control in Duckietown UGVs.]()
Figure 1. Spool CAD Design – Isometric View.
![Front view of a CAD model of a spool mechanism for flexible tether control in Duckietown robots.]()
Figure 2. Spool CAD Design – Front View.
![Physical implementation of the automated spool system for flexible tether control in Duckietown UGVs.]()
Figure 3. Physical Spool for Flexible Tether Control.
![Modified Duckiebot DB21J with an attached tether for flexible tether control in robotic testing.]()
Figure 4. Modified Duckiebot DB21J for Tether Control Testing.
![ROS graph illustrating data flow in a flexible tether control system for Duckietown UGVs.]()
Figure 5. ROS Graph for Tether Management System.
![Three different cases of tether length in a flexible tether control system: too long, too short, and correctly adjusted.]()
Figure 6. Tether Length Cases – Too Long, Too Short, Just Right.
![DB21J trial trajectory demonstrating the effect of flexible tether control on robot mobility.]()
Figure 7. DB21J Trial Trajectory with Tether Control.
![Figure 8. Wheel Velocities vs. Time for Multiple Trials.]()
Figure 8. Wheel Velocities vs. Time for Multiple Trials.
![Robot distance from the spool and tether error measurements in a flexible tether control system.]()
Figure 9. Robot Distance & Tether Error vs. Time.
![Measured tether length across different slackness values in a flexible tether control system.]()
Figure 10. Measured Tether Length vs. Slackness.
![Tether error measurements for different control gain values in a flexible tether control system.]()
Figure 11. Tether Error vs. Control Gain.
## Flexible tether control in Duckietown: objective and importance
Managing tethers effectively is an important challenge in autonomous robotic systems, especially in heterogeneous marsupial robot setups where multiple robots work together to achieve a task.
Tethers provide power and data connections between agents, but poor management can lead to tangling, restricted movement, or unnecessary strain.
This work implements a flexible tethering approach that balances slackness and tautness to improve system performance and reliability.
Using the [Duckiebot DB21J](https://get.duckietown.com/products/duckiebot-db21 "Get a Duckiebot (DB21J)") as a test passenger agent, the study introduces a tether control system that adapts to different conditions, ensuring smoother operation and better resource sharing. By combining aspects of both taut and slacked tether models, this work contributes to making multi-robot systems more efficient and adaptable in various environments.
[ Learn robot autonomy with Duckietown ](https://duckietown.com/educational-resources/#autonomy)
## The method and challenges in implementing flexible tether control in Duckietown
The authors developed a custom-built spool mechanism designed to actively adjust tether length using real-time sensor feedback. The tether system comprises a custom-built spool mechanism, integrated with sensor feedback for real-time tether length adjustments.
To coordinate these adjustments, the system was implemented within a standard ROS-based framework, ensuring efficient data management.
To evaluate the system’s effectiveness, the authors tested different slackness and control gain parameters while the Duckiebot followed a predefined square path. By analyzing the spool’s reactivity and the consistency of the tether’s behavior, they assessed the system’s performance across varying conditions.
Several challenges emerged during testing, e.g., maintaining the right balance of tether slackness was critical, as excess slack risked entanglement, while insufficient slack could restrict mobility.
Hardware limitations affected the spool’s responsiveness, requiring careful tuning of control parameters. Additionally, environmental factors, such as potential obstacles, underscored the need for a more adaptive control mechanism in future iterations.
## Flexible tether control: full report
Check out the full report here.
## Flexible tether control in heterogeneous marsupial systems in Duckietown: Authors
[  ](https://www.linkedin.com/in/carson-duffy-19940a18a/)
[Carson Duffy](https://www.linkedin.com/in/carson-duffy-19940a18a/) is a computer engineer who studied at the [Texas A&M University](https://www.tamu.edu/index.html), USA.
[  ](https://www.linkedin.com/in/jokane/)
[Dr. Jason O’Kane](https://www.linkedin.com/in/jokane/) is a faculty research advisor at Texas A&M.
### Learn more
Duckietown is a modular, customizable, and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
Duckietown is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
These spotlight projects are shared to exemplify Duckietown’s value for hands-on learning in robotics and AI, enabling students to apply theoretical concepts to practical challenges in autonomous robotics, boosting competence and job prospects.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Projects
**Tags:** project, student project
---
### [Ozgur Erkent: robotic rescue operations with Duckietown](https://duckietown.com/ozgur-erkent-exploring-robotics-and-rescue-operations-with-duckietown/)
**Published:** February 27, 2025
**Author:** Duckietown Admin
**Excerpt:** Meet Ozgur Erkent, Assistant Professor at Hacettepe University’s Computer Engineering Department in Turkey, who is teaching and doing research with Duckietown.
**Content:**
# Ozgur Erkent: robotic rescue operations with Duckietown
Meet Ozgur Erkent, Assistant Professor at Hacettepe University’s Computer Engineering Department in Turkey, who is teaching and doing research with Duckietown.
**Ankara, Turkey, January 2025:** Prof. Ozgur Erkent shares how Duckietown is shaping robotics education at Hacettepe University. From hands-on learning in his Introduction to Robotics course, to real-world applications in rescue operations, he explains why he believes Duckietown is an invaluable tool for students exploring autonomous systems.
##### Quick links
- [ Ozgur Erkent on Linkedin ](https://www.linkedin.com/in/ozgurerkent/?originalSubdomain=tr)
- [ Hacettepe University Ankara ](https://www.hacettepe.edu.tr/english)
- [ Introduction to Robotics course - Prof.Erkent ](https://web.cs.hacettepe.edu.tr/~ozgurerkent/RoboticsUnderGrad.html)
- [ Hacettepe Robotics Lab ](https://web.cs.hacettepe.edu.tr/~ozgurerkent/RoboticsLab.html)
- [ Bridge to Turkiye Fund ](https://bridgetoturkiye.org/)
## Bringing hands-on robotics to the classroom
At Hacettepe University, Professor Ozgur Erkent is using Duckietown in his curriculum and providing students with hands-on learning experiences that bridge theory and real-world applications.
##### Good morning and welcome! Could you introduce yourself and your work?
My name is Ozgur Erkent and I am an Assistant Professor at Hacettepe University’s Computer Engineering Department. I have been here for nearly three years, focusing on mobile robots and autonomous vehicles. My work involves both teaching and research in these areas.
##### How did you first discover Duckietown?
I first heard about Duckietown while working as a researcher in France. A colleague returning from Colombia shared how undergraduates were using Duckiebots in their projects. That caught my interest, and when I joined Hacettepe University, I saw an opportunity to integrate it into my courses.
##### What course do you use Duckietown for, and what does it involve?
I use Duckietown in my Introduction to Robotics course, which is open to third- and fourth-year students in the Artificial Intelligence Engineering program. The course has a laboratory component where students work with Duckiebots and Duckiedrones to apply robotics concepts practically.
I also wrote a project funded by NVIDIA through the “Bridge To Turkiye Fund”, that focuses on rescue robotics. After the devastating earthquake in Turkey two years ago, NVIDIA launched an initiative to support research aimed at disaster response. With NVIDIA as the sponsor, we were able to purchase the Duckiebots, Duckiedrones and related tools for the Robotics Lab course. I proposed a project that leverages Duckietown kits to train students in **SLAM** (Simultaneous Localization and Mapping), sensor integration, and autonomous navigation—key skills for robotics applications in search and rescue operations. Through this project, students may gain hands-on experience in developing robotic systems that could one day assist in real-world disaster relief efforts.
> Robotics is more than just algorithms; it’s about solving real-world challenges. Duckietown helps students bridge that gap in a meaningful way.
>
> Ozgur Erkent
##### How have students reacted to working with Duckietown?
Many students come from a software background, so working with real hardware is a new challenge. Some find it difficult at first, but those who enjoy hands-on work really thrive. They even help their peers with assembly and troubleshooting. It’s a valuable learning experience. If I were to design something for undergraduate students learning robotics, it would probably look a lot like Duckietown. I think it would be a great addition, as it would help students get hands-on experience with the basics of robotics.
> If I were to design something for undergraduate students learning robotics, it would probably look a lot like Duckietown. I think it would be a great addition, as it would help students get hands-on experience with the basics of robotics.
>
> Ozgur Erkent
##### Besides Duckiebots, are you using any other tools?
Yes, I have also introduced [Duckiedrones](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24), which are especially popular in Turkey. The national foundation supports drone projects, and students are eager to explore them. Several groups are already working on Duckiedrone-based initiatives.
##### What do you think about the Duckietown community and support?
The community is a big advantage. Universities considering Duckietown should definitely check out its forums and resources. The support available makes a big difference in implementing the platform effectively.
##### Any final thoughts?
I’m excited to see where these projects lead. Robotics is more than just algorithms; it’s about solving real-world challenges. Duckietown helps students bridge that gap in a meaningful way.
### Learn more about Duckietown
Duckietown enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** News, People
**Tags:** AI, duckiebot, rescue operations, robotics
---
### [PID and Convolutional Neural Networks (CNN) in Duckietown](https://duckietown.com/pid-and-convolutional-neural-network-cnn-in-duckietown/)
**Published:** February 8, 2025
**Author:** Duckietown Admin
**Excerpt:** This paper explores PID and convolutional neural networks (CNN) for autonomous driving, comparing their performance for line-following tasks in Duckietown.
**Content:**
##### General Information
- **Title**: Application of PID controller and CNN to control Duckiebot robot
- **Authors**: Marek Długosz, Paweł Skruch, Marcin Szelest, Artur Morys-Magiera
- **Institution**: AGH University of Science and Technology, Poland
- **Citation**: M. Długosz, P. Skruch, M. Szelest and A. Morys-Magiera, "Application of PID controller and CNN to control Duckiebot robot," 2023 21st International Conference on Emerging eLearning Technologies and Applications (ICETA), Stary Smokovec, Slovakia, 2023, pp. 105-110, doi: 10.1109/ICETA61311.2023.10344003.
[ Paper ](https://ieeexplore.ieee.org/document/10344003)
[ Authors ](#authors)
[ Institution ](https://www.agh.edu.pl/en/)
# PID and Convolutional Neural Networks (CNN) in Duckietown
Ever wondered how the legendary **PID controller** compares to a more “modern” convolutional neural network (**CNN**) design, in controlling a Duckiebot in driving in Duckietown?
This work analyzes the **performance differences** between classical control techniques and machine learning-based approaches for autonomous navigation. The Duckiebot follows a designated path using image-based feedback, where the PID controller corrects deviations through proportional, integral, and derivative adjustments. The CNN-based method leverages image feature extraction to generate control commands, reducing reliance on predefined system models.
Key aspects covered include differential **drive mechanics**, **real-time image processing**, and **ROS**-based implementation. The study also outlines the impact of training data selection on CNN performance. Comparative analysis highlights the strengths and limitations of both approaches. The conclusions emphasize the applicability of PID and CNN techniques in Duckietown, demonstrating their role in advancing robotic autonomy.
[ Learn about machine learning with Duckietown materials ](https://duckietown.com/educational-resources/)
## Highlights - PID and Convolutional Neural Network (CNN) in Duckietown
Here is a **visual tour** of the work of the authors. For all the details, check out the [full paper](#links "Post resources").
![PID-controlled Duckiebot robot used for autonomous navigation in Duckietown]()
Figure 1. Duckiebot in Duckietown.
![PID-based kinematic system of Duckiebot in Duckietown]()
Figure 2. Kinematic Model of Duckiebot in Duckietown.
![PID controller schematic for Duckiebot navigation in Duckietown]()
Figure 3. PID-Based Control System for Duckiebot in Duckietown.
![PID-based error detection for Duckiebot line following in Duckietown]()
Figure 4. Error Calculation for PID Line Following in Duckiebot.
![PID-controlled Duckiebot image processing in Duckietown for position error detection]()
Figure 5. Image Processing for PID Error Detection in Duckiebot.
![PID alternative CNN-based Duckiebot control system in Duckietown]()
Figure 6. CNN-Based Control System for Duckiebot in Duckietown.
![PID-free CNN architecture for Duckiebot control in Duckietown]()
Figure 7. CNN Architecture for Duckiebot in Duckietown.
![Histogram of PID and CNN training data for Duckiebot control in Duckietown]()
Figure 8. Data Distribution for Duckiebot Training in Duckietown.
![PID versus CNN image preprocessing for Duckiebot control in Duckietown]()
Figure 9. Image Preprocessing for CNN-Based Duckiebot Control in Duckietown.
![PID and CNN control development setup for Duckiebot in Duckietown]()
Figure 10. Duckiebot Development Environment for PID and CNN Control.
## Abstract
In the author’s words:
The paper presents the design and practical **implementation by students of a control system** using a classic **PID controller** and a controller using **artificial neural networks**. The control object is a Duckiebot robot, and the task it is to perform is to drive the robot along a designated line (line follower).
The **purpose** of the proposed activities is to **familiarize students** with the advantages and disadvantages of the two controllers used and for them to acquire the **ability to implement** control systems in practice. The article briefly describes how the two controllers work, how to practically implement them, and how to practically implement the exercise.
## Conclusion - PID and Convolutional Neural Network (CNN) in Duckietown
Here are the conclusions from the author of this paper:
“The **PID controller** is used successfully in many control systems, and its implementation is relatively **simple**. There are also a number of methods and algorithms for adjusting controller parameters for this type of controller.
PID controllers, on the other hand, are **not free of disadvantages**. One of them is the requirement of prior knowledge of, even roughly, the model of the process one wants to control. Thus, it is necessary to **identify** both the structure of the process **model** and its parameters. Identification tasks are complex tasks, requiring a great deal of knowledge about the nature of the process itself. There are also methods for identifying process models based on the results of practical experiments, however sometimes it may not be possible to conduct such experiments. When using a PID controller, one should also be aware that it was developed for processes, operation of which can be described by linear models. Unfortunately, the behavior of the vast majority of dynamic systems is described by non-linear models.
The consequence of this fact is that, in such cases, the PID controller works using **linear approximations** of nonlinear systems, which can lead to various errors, inaccuracies, etc. Unlike the classic PID controller, controllers using artificial neural networks do not need to know the mathematical model of the process they control and its parameters.
The ability to design different neural network architectures, such as **convolutional, recurrent, or deep neural** networks, makes it possible to adapt the neural regulator to the specific process it is supposed to control. On the other hand, the multiplicity of neural network architectures and their design means that we can never be sure whether a given neural network structure is optimal.
The selection of neural controller parameters is done automatically using appropriate **network training algorithms**. The key element influencing the accuracy of neural regulator operation is the data used for training the neural network. The disadvantage of regulators using neural networks is the inability to demonstrate the stability of operation of the systems they control.
In case of the PID regulator, despite the use of approximate models of the process, it is very often possible to prove that a closed control system will operate stably in any or a certain range of values of variables. Unfortunately, such an analysis cannot be carried out in the case of neural regulators. In summary, the implementation of two different controllers to perform the same task provides an opportunity to learn the advantages and disadvantages of each.”
#### Project Authors
[  ](https://scholar.google.pl/citations?hl=pl&user=XFIEzpsRqNIC)
[Marek Długosz](https://scholar.google.pl/citations?hl=pl&user=XFIEzpsRqNIC) is a Professor at the [Akademia Górniczo-Hutnicza (AGH)](https://www.agh.edu.pl/en/) – University of Science and Technology, Poland.
[  ](https://www.linkedin.com/in/pawe%C5%82-skruch-6a44b987/?originalSubdomain=pl)
[Paweł Skruch](https://www.linkedin.com/in/pawe%C5%82-skruch-6a44b987/?originalSubdomain=pl) is currently working as the Manager and Principal Engineer AI at [Aptiv](http://www.aptiv.com/), Switzerland.
[  ](https://ieeexplore.ieee.org/author/37887798800)
[Marcin Szelest](https://ieeexplore.ieee.org/author/37887798800) is currently affiliated with the [AGH University of Krakow](https://www.agh.edu.pl/en/), Kracow, Poland.
[  ](https://artus9033.vercel.app/)
[Artur Morys-Magiera](https://artus9033.vercel.app/) is a a PhD candidate at [AGH University of Krakow](https://www.agh.edu.pl/en/), Poland.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** research
---
### [Visual monitoring of automated guided vehicles in Duckietown](https://duckietown.com/visual-monitoring-of-automated-guided-vehicles-in-duckietown/)
**Published:** January 25, 2025
**Author:** Duckietown Admin
**Excerpt:** This work implemented a visual monitoring system in Duckietown to enable real-time trajectory tracking and enhance traffic safety.
**Content:**
##### General Information
- **Title**: Visual Monitoring of Swarms of Industrial Robots
- **Authors**: Anastasia Kravchenko, Alexey Sychev, Vladimir Zyubin
- **Institution**: Institute of Automation and Electrometry, Russia
- **Citation**: A. Kravchenko, A. Sychev and V. Zyubin, "Visual Monitoring of Swarms of Industrial Robots," 2023 International Russian Automation Conference (RusAutoCon), Sochi, Russian Federation, 2023, pp. 604-609, doi: 10.1109/RusAutoCon58002.2023.10272883.
[ Paper ](https://ieeexplore.ieee.org/abstract/document/10272883)
[ Authors ](#authors)
[ Institution ](https://www.iae.nsk.su/en/)
# Visual monitoring of automated guided vehicles in Duckietown
The increasing use of robotics in industrial automation has led to the need for systems that ensure safety and efficiency in monitoring autonomous guided vehicles (AGVs). This research proposes a visual monitoring system for monitoring the trajectory and behavior of AGVs in industrial environments.
The system utilizes a network of cameras mounted on towers to detect, identify, and track AGVs. The visual data is transmitted to a central server, where the robots’ trajectories are evaluated and compared against predefined ideal paths. The system operates independently of specific hardware or software configurations, offering flexibility in its deployment.
Duckietown was used as the test environment for this system, allowing for controlled experiments with simulated robotic fleets. A prototype of the system demonstrated its capability to track AGVs using Aruco tags and evaluate rectilinear trajectories.
Key aspects and concepts:
- Use of camera towers for visual control of AGVs;
- Transmission of visual data to a central server for trajectory evaluation;
- Compatibility with multiple robot types and operating systems;
- Integration of Aruco tags for robot identification;
- Modular architecture enabling future expansions;
- Testing in Duckietown for controlled evaluation.
This research demonstrates a modular approach to monitoring AGVs using a visual control system tested in the Duckietown platform. Future work will extend the system’s capability to handle more complex trajectories such as turns and arcs, further leveraging Duckietown as a scalable research and testing environment.
[ Learn about machine learning with Duckietown materials ](https://duckietown.com/educational-resources/)
## Highlights - Visual monitoring of automated guided vehicles in Duckietown
Here is a visual tour of the work of the authors. For all the details, check out the [full paper](#links "Post resources").
![Diagram showing the general scheme of the visual monitoring system for monitoring AGVs, integrated with Duckietown.]()
Figure 1. General Scheme of the Visual Monitoring System.
![Diagram showing the architecture of the visual monitoring system, including Duckietown integration for AGV fleet monitoring.]()
Figure 2. Architecture of the Visual Monitoring System.
![Block diagram of the visual control system with Duckietown integration, showing components and their interactions.]()
Figure 3. Block Diagram of the Visual Monitoring System.
![A Duckiebot with an attached Aruco tag, used for identification and tracking within the visual control system integrated with Duckietown.]()
Figure 4. Duckiebot with Aruco Tag for Identification.
![Diagram of the algorithm used in the visual control system, integrating Duckietown for AGV tracking and monitoring.]()
Figure 5. Algorithm Scheme for Visual Monitoring System.
![Test setup of the visual control system in operation, featuring Duckietown integration for monitoring AGVs.]()
Figure 6. Test Setup of the Visual Monitoring System in Action.
![Example image captured by the camera in the visual control system, tracking AGVs in the Duckietown environment.]()
Figure 7. Sample Camera Image from Visual Monitoring System.
![Input images fed into the visual control system with Duckietown integration for AGV detection and tracking.]()
Figure 8. Input Images for the Visual Monitoring System.
![Results from the visual control system, including AGV trajectories and monitoring data in the Duckietown environment.]()
Figure 9. Visual Monitoring System Results.
## Abstract
In the author’s words:
With the increasing automation of industry and the introduction of robotics in every step of the production chain, the problem of safety has become acute. The article proposes a solution to the problem of safety in production using a visual control system for the fleet of loading automated guided vehicles (AGV). The visual control system is built as towers equipped with cameras. This approach allows to be independent of equipment vendors and allows flexible reconfiguration of the AGV fleet. The cameras detect the appearance of a loading robot, identify it and track its trajectory. Data about the robots’ movements is collected and analyzed on a server. A prototype of the visual control system was tested with the Duckietown project.
## Conclusion - Visual monitoring of automated guided vehicles in Duckietown
Here are the conclusions from the author of this paper:
“In the course of this work, a prototype visual evaluation system for Duckietown project was implemented. The system supports flexible seamless integration of third-party detection algorithms and trajectory evaluation algorithms. The visual control system was tested with client imitator module, witch does not require the presence of the real robot on the field. At this stage of the work, the prototype is able to recognize rectilinear trajectory of motion. In the future, we plan to develop evaluation algorithms for other types of trajectories: 90 degree turns, large angle turns, arc movement, etc. Another promising area of research is the integration of the system with cloud-based integrated development environments (IDEs) for industrial control algorithms.”
#### Project Authors
[  ](https://www.linkedin.com/in/michael-yuhas-091b347a/?originalSubdomain=sg)
[Anastasia Kravchenko](https://ieeexplore.ieee.org/author/37090035699) is currently affiliated to Department of Cyber Physical Systems [Institute of Automation and Electrometry](https://www.iae.nsk.su/en/) SB RAS Novosibirsk, Russia.
[  ](https://www.linkedin.com/in/michael-yuhas-091b347a/?originalSubdomain=sg)
[Alexey Sychev](https://ieeexplore.ieee.org/author/37090036585) is currently affiliated to Department of Cyber Physical Systems [Institute of Automation and Electrometry](https://www.iae.nsk.su/en/) SB RAS Novosibirsk, Russia.
[  ](https://www.linkedin.com/in/arvind-easwaran-066544292/?originalSubdomain=sg)
[Vladimir Zyubin](https://ieeexplore.ieee.org/author/37697825900) is currenly working as an Associate Professor at the [Institute of Automation and Electrometry](https://www.iae.nsk.su/en/), Russia.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** research
---
### [Intelligent and autonomous mobility systems](https://duckietown.com/intelligent-and-autonomous-mobility-systems/)
**Published:** January 13, 2025
**Author:** Federico Tani
**Excerpt:** Research Associates Yikai Zeng and Xinyu Zhang from the Technische Universität Dresden tell us about their work in developing autonomous mobility systems.
**Content:**
# Intelligent and autonomous mobility systems
Research Associates Yikai Zeng and Xinyu Zhang from the Technische Universität Dresden tell us about their work in developing autonomous mobility systems.
**Dresden, Germany, November 22, 2024:** Research Associates Yikai Zeng and Xinyu Zhang talk with us about the future of autonomous mobility and intelligent transportation systems that promise to redefine how we think about movement and connectivity in urban spaces.
##### Quick links
- [ Yikai Zeng on Linkedin ](https://www.linkedin.com/in/yikaizeng/?locale=en_US)
- [ Xinyu Zhang on Linkedin ](https://www.linkedin.com/in/xinyu-zhang-2028741bb/?original_referer=&originalSubdomain=de)
- [ Technische Universität Dresden - TUD ](https://tu-dresden.de/)
- [ MiniCCAM Lab ](https://tu-dresden.de/bu/verkehr/vis/vpa/labore/miniature-connected-cooperative-and-automated-mobility-lab?set_language=enstem/summer-drone-acadamy/)
- [ Chair of Traffic Process Automation - TUD ](https://tu-dresden.de/bu/verkehr/vis/vpa/startseite?set_language=en)
- [ An Introduction to the Chair of Traffic Process Automation ](https://ieeexplore.ieee.org/document/10462526)
## Connected, cooperative and autonomous mobility
We talked with Yikai Zeng and Xinyu Zhang from the Chair of Traffic Process Automation at TU Dresden about their research and teaching activities, and how Duckietown is used at the MiniCCAM lab to teach autonomous mobility.
##### Hello and welcome! May I ask you to start by introducing yourself?
**X. Zhang**: Hi! I will start! My name is Xinyu. I’m a Research Associate at TU Dresden and currently work on computational basics and tools of traffic process automation. That’s why I got involved in this [Duckiedrone](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24) demonstration. Apart from that, I am also responsible for the basic autonomous driving courses, where we use Duckiebots as our learning materials and tools for the students.
**Y. Zeng**: Hello, My name is Yikai. I’m also a Research Associate at TU Dresden, in Prof. Meng Wang’s laboratory.
##### Thank you very much, when did you first discover Duckietown?
**Y. Zeng**: the idea came from Professor Wang, who asked us to continue the Control course of a former colleague, using among other things, Duckiebots. When we took over the course, it was during the Covid period. Right now we have developed the MiniCCAM lab.
##### Could you tell us more about the miniCCAM lab?
**Y. Zeng**: Sure! The scope of the miniCCAM laboratory, for us researchers in the transportation and autonomous mobility field, is to look at the greater picture in terms of urban mobility, so slightly different in terms of scope than the course previously mentioned. We use Duckietown for autonomous driving. The current miniCCAM lab is on one hand a good tool for demonstrating to students and general audiences what we are able to do in terms of future transportation systems; on the other hand, it provides us with an opportunity to conduct research. For example, we implemented a higher logic controller for intersection navigation and tested it in both a simulated environment and on the model smart-city Duckietown setup. Duckietown is very practical because organizing an actual field test would be very expensive.
##### That's great to hear. Why did you decide to use Duckiebots to teach autonomous mobility?
**Y. Zeng**: The decision was taken before us, but I heard stories about that time. So this course has a long history, over ten years, and every few years the course was redesigned.
Around 2019 the decision was taken to upgrade our fleet of robots, and among various solutions, we also chose Lego initially, but it didn’t work very well for us.
So my former colleague found out about Duckietown, and that’s when the choice was taken. It came all in a single box, and this was considered very positive. It also came with complete teaching materials and very well-structured courses already. This was considered to be extremely useful to help us organize our courses, we just needed to modify what was already there for our own context. So this would be the main motivation, it’s very easy to deploy course materials, and the economic aspects were considered to be very attractive.
**X. Zhang**: Duckiebots are also good because they come with a camera and wheel encoders, making it easier to get students started, and having them learn about the fundamentals of autonomous driving.
> It came all in a single box, complete teaching materials and very well-structured courses. This was considered to be extremely useful, we just needed to modify what was already there for our own context.
>
> Yikai Zeng
##### Did students appreciate using Duckiebots?
**Y. Zeng**: Certainly Duckietown succeeded as a teaching tool, attracting many students to our courses. I would say Duckietown has this characteristic of motivating and capturing the attention of many students. It also provides the first real hands-on experience in the field of robotics and autonomous mobility.
In our course on Computational basics and tools of traffic process automation ([Rechentechnische Grundlagen und Werkzeuge der Verkehrsprozessautomatisierung](https://tu-dresden.de/bu/verkehr/vis/vpa/studium/lehrveranstaltungen/beschreibung_rgw.html)), we use Duckiebots to teach students about general control, group control, and swarm control. Duckietown is also the main, shall we say, “tourist attraction” of our department. Every time we hold events, many students come to us to see the Duckiebots cooperating, going through intersections, and so forth. We’ve been using Duckietown for two years, and already it is very popular, inspiring many interesting discussions with our audiences with scientific backgrounds.
Much more efficient than a simple presentation, I’d say!
> Duckiebots come with a camera and wheel encoders, making it easier to get students started, and having them learn about the fundamentals of autonomous mobility.
>
> Xinyu Zhang
##### Would you recommend Duckietown to colleagues and students?
**Y. Zeng**: Yes absolutely, in fact, I’m a bit sad that you’re not producing the old model anymore! We definitely want to try the latest models, test them as a fleet, and introduce them to our lab in the future. Our main focus is always on the interaction between groups of bots and how they work together.
### Learn more about Duckietown
Duckietown enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** News, People
**Tags:** AI, duckiebot, miniCCAM, robotics
---
### [Variational Autoencoder for autonomous driving in Duckietown](https://duckietown.com/variational-autoencoder-for-autonomous-driving-in-duckietown/)
**Published:** December 6, 2024
**Author:** Duckietown Admin
**Excerpt:** This research explores using variational autoencoder and reinforcement learning (RL) for lane following in Duckietown, highlighting benefits and challenges.
**Content:**
##### General Information
- **Title**: Learning to Drive with Reinforcement Learning and Variational Autoencoders
- **Authors**: Bryon Kucharski
- **Institution**: University of Massachusetts Amherst, United States
- **Citation**: Kucharski, B., Learning to Drive with Reinforcement Learning and Variational Autoencoders.
[ Paper ](https://bryonkucharski.github.io/files/RL_VAE_final_report.pdf)
[ Code ](https://github.com/bryonkucharski/Learning-to-Drive-with-Reinforcement-Learning-and-Variational-Autoencoders)
[ Institution ](https://www.umass.edu/)
[ Authors ](#authors)
# Variational Autoencoder for autonomous driving in Duckietown
This project explored using reinforcement learning (RL) and Variational Autoencoder (VAE) to train an autonomous agent for lane following in the Duckietown Gym simulator. VAEs were used to encode high-dimensional raw images into a low-dimensional latent space, reducing the complexity of the input for the RL algorithm (Deep Deterministic Policy Gradient, DDPG). The goal was to evaluate if this dimensionality reduction improved training efficiency and agent performance.
The agent successfully learned to follow straight lanes using both raw images and VAE-encoded representations. However, training with raw images performed similarly to VAEs, likely because the task was simple and had limited variability in road configurations.
The agent also displayed discrete control behaviors, such as extreme steering, in a task requiring continuous actions. These issues were attributed to the network architecture and limited reward function design.
While the VAE reduced training time slightly, it did not significantly improve performance. The project highlighted the complexity of RL applications, emphasizing the need for robust reward functions and network designs.
[ Learn about machine learning with Duckietown materials ](https://duckietown.com/educational-resources/#machine-learning)
## Highlights - Variational Autoencoder and RL for Duckietown Lane Following
Here is a visual tour of the work of the authors. For all the details, check out the [full paper](#links "Post resources").
![A screenshot showing examples of the Gym-Duckietown simulator, featuring a virtual environment with roads, lanes, and small robotic vehicles.]()
Figure 1. Examples of the Gym-Duckietown Simulator Environment.
![A series of images showing the reconstruction capabilities of a Variational Autoencoder (VAE) improving from the start to the end of training.]()
Figure 2. Progression of VAE Reconstruction from Start to End of Training.
![A top-down view of the Gym-Duckietown map used in experiment 1, featuring roads, intersections, and marked lanes for autonomous driving simulations.]()
Figure 3. Gym-Duckietown Map Used in Experiment 1.
![A plot showing the average reward per iteration during training for experiment 1, averaged over 10 trials. The goal is an average reward near 1.0.]()
Figure 4. Experiment 1 Results: Average Reward per Training Iteration.
![A top-down view of the Gym-Duckietown map used in experiment 2, featuring more complex road layouts and lane configurations for autonomous driving tasks.]()
Figure 5. Gym-Duckietown Map Used in Experiment 2.
![A plot showing the training reward for a single trial of experiment 2, indicating that the agent fails to achieve a positive reward despite staying in the middle of the lane on straight sections.]()
Figure 6. Training Reward for Single Trial of Experiment 2.
## Abstract
In the author’s words:
The use of deep reinforcement learning (RL) for following the center of a lane has been studied for this project. Lane following with RL is a push towards general artificial intelligence (AI) which eliminates the use for hand crafted rules, features, and sensors.
A project called **Duckietown** has created the Artificial Intelligence Driving Olympics, which aims to promote AI education and embodied AI tasks. The AIDO team has released an open-sourced simulator which was used as an environment for this study. This approach uses the Deep Deterministic Policy Gradient (DDPG) with raw images as input to learn a policy for driving in the middle of a lane for two experiments. A comparison was also done with using an encoded version of the state as input using a Variational Autoencoder (VAE) on one experiment.
A variety of reward functions were tested to achieve the desired behavior of the agent. The agent was able to learn how to drive in a straight line, but was unable to learn how to drive on curves. It was shown that the VAE did not perform better than the raw image variant for driving in the straight line for these experiments. Further exploration of reward functions should be considered for optimal results and other improvements are suggested in the concluding statements.
## Conclusion - Variational Autoencoder and RL for Duckietown Lane Following
Here are the conclusions from the author of this paper:
“After the completion of this project, I have gained insight on how difficult it is to get RL applications to work well. Most of my time was spent trying to tune the reward function. I have a list of improvements that are suggested as future work.
- Different network architectures – I used fully connected networks for all the architectures. I would think CNN architectures may be better at creating features for state representations.
- Tuning Networks – Since most of my time was spent on the reward exploration, I did not change any parameters at all. I followed the paper in the original DDPG paper \[4\]. A hyperparameter search may prove to be beneficial to find parameters that work best for my problem instead of all the problems in the paper.
- More training images for VAE
- Different Algorithm – Maybe an algorithm like PPO may be able to learn a better policy?
- Linear Function Approximation – Deep reinforcement learning has proven to be difficult to tune and work well. Maybe I could receive similar or better results using a different function approximator than a neural network. Wayve explains the use of prioritized experience replay \[7\], which is a method to improve on randomly sampled tuples of experiences during RL training and is based on sorting the tuples. This may improve performance of both of my algorithms.
- Exploring different Ornstein-Uhlenbeck process parameters to encourage, discourage more/less exploration
- Other dimensionality reducing methods instead of VAE. Maybe something like PCA?
As for the AIDO competition, I have made the decision not to submit this work. It became apparent to me as I progressed through the project how difficult it is to get a perfectly working model using reinforcement learning. If I was to continue with this work for the submission, I think I would rather go towards the track of imitation learning. While this would introduce a wide range of new problems, I think intuitively it moves more sense to ”show” the robot how it should drive on the road rather having it learn from scratch. I even think classical control methods may work better or just as good as any machine learning based algorithm. Although I will not submit to this competition, I am glad I got to express two interests of mine in reinforcement learning and variational autoencoders.
The supplementary documents for this report include the training set for the VAE, a video of experiment 1 working properly for both DDPG+Raw and DDPG+VAE, and a video of experiment 2 not working properly. The code has been posted to GitHub ([Click for link](https://github.com/bryonkucharski/Learning-to-Drive-with-Reinforcement-Learning-and-Variational-Autoencoders)).”
#### Project Authors
[  ](https://www.linkedin.com/in/bryonkucharski/)
[Bryon Kucharski](https://www.linkedin.com/in/bryonkucharski/) is currently working as a Lead Data Scientist at [Gartner](https://www.gartner.com/en), United States.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** education, networks, research
---
### [Autonomy Education: Teaching Networked Systems](https://duckietown.com/networked-systems-autonomy-education-with-duckietown/)
**Published:** November 16, 2024
**Author:** Duckietown Admin
**Excerpt:** This research explores how networked systems education is enhanced with Duckietown, a platform offering hands-on autonomy learning for students worldwide.
**Content:**
##### General Information
- **Title**: On the Education of Networked Systems
- **Authors**: Qing-Shan Jia
- **Institution**: Tsinghua University, Beijing, China
- **Citation**: Q. S. Jia, "On the Education of Networked Systems," 2022 41st Chinese Control Conference (CCC), Hefei, China, 2022, pp. 7572-7577, doi: 10.23919/CCC55666.2022.9902623.
[ Paper ](https://ieeexplore.ieee.org/document/9902623)
[ Institution ](https://www.tsinghua.edu.cn/en/)
[ Authors ](#authors)
# Autonomy Education: Teaching Networked Systems
In this work, Prof. Qing-Shan Jia from Tsinghua University in China explores the challenges and innovations in teaching networked systems, a domain with applications ranging from smart buildings to autonomous systems.
The study reviews curriculum structures and introduces practical solutions developed by the Tsinghua University Center for Intelligent and Networked Systems (CFINS).
Over the past two decades, CFINS has designed courses, developed educational platforms, and authored textbooks to bridge the gap between theoretical knowledge and practical application.
They feature Duckietown as part of an educational platform for autonomous driving. Duckietown offers a low-cost, do-it-yourself (DIY) framework for students to construct and program Duckiebots – autonomous mobile robotic vehicles. Duckietown allows learners to apply theoretical concepts in areas related to robot autonomy, like signal processing, machine learning, reinforcement learning, and control systems.
Duckietown enables students to gain hands-on experience in systems engineering, with calibration of sensors, programming navigation algorithms, and working on cooperative behaviors in multi-robot settings. This approach allows for the creation of complex cyber physical systems using state-of-the-art science and technology, not only democratizing access to autonomy education but also fostering understanding, even with remote learning scenarios.
The integration of Duckietown into the curriculum exemplifies the innovative strategies employed by CFINS to make networked systems education both practical and impactful.
[ Learn about robot autonomy with Duckietown materials ](https://duckietown.com/educational-resources/#autonomy)
[](https://duckietown.com/wp-content/uploads/2024/11/image.avif) Figure 1. Education Platform for Smart Buildings. [](https://duckietown.com/wp-content/uploads/2024/11/image-1.avif) Figure 2. Human-Machine Interface for Indoor Environments by Tsinghua University.
[](https://duckietown.com/wp-content/uploads/2024/11/image-2.avif) Figure 3. Smart Building Evacuation Guidance with Networked Systems. [](https://duckietown.com/wp-content/uploads/2024/11/image-3.avif) Figure 4. Education Platform for Networked Manufacturing Systems.
## Abstract
In the author’s words:
Networked systems have become pervasive in the past two decades in modern societies. Engineering applications can be found from smart buildings to smart cities. It is important to educate the students to be ready for designing, analyzing, and improving networked systems.
But this is becoming more and more challenging due to the conflict between the growing knowledge and the limited time in the curriculum. In this work we consider this important problem and provide a case study to address these challenges.
A group of courses have been developed by the Center for Intelligent and Networked Systems, department of Automation, Tsinghua University in the past two decades for undergraduate and graduate students. We also report the related education platform and textbook development. Wish this would be useful for the other universities.
## Conclusion - Networked Systems: Autonomy Education with Duckietown
Here are the conclusions from the author of this paper:
“In this work we provided a case study on the education practice of networked systems in the center for intelligent and networked systems, department of automation, Tsinghua University. The courses mentioned in this work have been delivered for 20 years, or even more. From this education practice, the following experience is summarized. First, use research to motivate the study.
Networked systems is a vibrant research field. The exciting applications in smart buildings, autonomous driving, smart cities serve as good examples not just to motivate the students but also to make the teaching materials concrete. Inviting world-class talks and short-courses are also good practice. Second, education platforms help to learn the knowledge better. Students have hands-on experience while working on these education platforms.
This project-based learning provides a comprehensive experience that will get the students ready for addressing the real-world engineering problems. Third, online/offline hybrid teaching mode is new and effective. This is especially important due to the pandemic. Lotus Pond, RainClassroom, and Tencent Meeting have been well adopted in Tsinghua. Students can interact with the teachers more frequently and with more specific questions.
They can also replay the course offline, including their answers to the quiz and questions in the classroom. We hope that this summary on the education on networked systems might help the other educators in the field.”
#### Project Authors
[  ](https://www.linkedin.com/in/samuel-qing-shan-jia-70037560/?originalSubdomain=ca)
[Qing-Shan Jia](https://www.linkedin.com/in/samuel-qing-shan-jia-70037560/?originalSubdomain=ca) is a Professor at the [Tsinghua University](https://www.tsinghua.edu.cn/en/), Beijing, People’s Republic of China.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** education, networks, research
---
### [Autonomous Calibration - Wheels and Camera in Duckietown](https://duckietown.com/autonomous-calibration-wheels-and-camera-in-duckietown/)
**Published:** October 31, 2024
**Author:** Duckietown Admin
**Excerpt:** This research presents a fully autonomous calibration approach for camera and wheels for the Duckiebots, by using environmental data and self-movement.
**Content:**
##### General Information
- **Title**: Autonomous Wheels And Camera Calibration In Duckietown Project
- **Authors**: Kirill Krinkin, Konstantin Chayka, Anton Filatov, Artyom Filatov
- **Institution**: Saint Petersburg Electrotechnical University, Russia
- **Citation**: Krinkin, K., Chayka, K., Filatov, A. and Filatov, A., 2021. Autonomous wheels and camera calibration in duckietown project. Procedia Computer Science, 186, pp.169-176.
[ Paper ](https://www.sciencedirect.com/science/article/pii/S1877050921009509)
[ Institution ](https://etu.ru/)
[ Authors ](#authors)
# Autonomous Calibration – Wheels and Camera in Duckietown
In robotics, accurate calibration of components like cameras and wheels is essential for precise operation. This research is focused on developing an autonomous calibration system for Duckiebots image sensors and odometry.
Traditional calibration methods require manual intervention, often taking time and relying on human accuracy, which can introduce variability. The paper presents a fully autonomous approach to calibration, enabling Duckiebots to perform self-calibration without human guidance. This enables users to calibrate multiple robots simultaneously, maximizing efficiency and reducing downtime.
Fiducial markers (AprilTags) are utilized in pre-marked environments. Although the method showed slightly reduced calibration precision compared to typical alternatives, the process still yields sufficient performance for Duckiebots to navigate autonomously in Duckietown.
[ Learn about calibrations with Duckietown materials ](https://duckietown.com/educational-resources/)
## Highlights - Autonomous Calibration - Wheels and Camera in Duckietown
Here is a visual tour of the work of the authors. For all the details, check out the [full paper](#links "Post resources").
![Initial position of a robot Duckiebot for autonomous calibration.]()
Figure 1. Initial position of the Duckiebot.
![State to state algorithm of calibraion.]()
Figure 2. State to state algorithm of calibraion.
![Reprojection error and straight line deviations]()
Figure 3. Reprojection error and straight line deviations.
## Abstract
In the author’s words:
After assembling the robot, it is necessary to calibrate its components such as camera and wheels for example. This requires human participation and depends on human factors. The article describes the approach to fully automatic calibration of the camera and the wheels of the robot.
It consists in placing the robot in an inaccurate position, but in a pre-marked area and using data from the camera, information about the configuration of the environment. As well as the ability to move, to perform calibration without the participation of external observers or human participation. There are 2 stages: camera and wheels calibration.
Camera calibration collects the necessary set of images by automatically moving the robot in front of the fiducial markers template, and moving the robot on the marked floor with an estimation of the curvature of the trajectory. Proposed approach was experimentally tested on the duckietown project base.
## Conclusion - Autonomous Calibration - Wheels and Camera in Duckietown
Here are the conclusions from the authors of this paper:
“As a result, a solution was developed that allows fully automatic calibration of the camera and robot wheels in the Duckietown project. The main feature is the autonomy of the process, which allows one person to run in parallel the calibration of an arbitrary number of robots and not be blocked during their calibration.
The limitation is the number of physically labeled sites. According to the results of comparing the developed solution with the initial one, a slight deterioration in accuracy can be noted, which is primarily associated with the accuracy of the camera calibration, however, the result obtained is nevertheless sufficient for the initial calibration of the robot and is comparable to manual calibration.
As the planned improvements, which will have to increase the accuracy of the camera calibration, a larger number of chessboards located at different angles and a greater distance of movement used in calibrating the wheels will be used.”
#### Project Authors
[  ](https://www.linkedin.com/in/krinkin/)
[Kirill Krinkin](https://www.linkedin.com/in/krinkin/) is an Adjunct Professor at [Constructor University](https://constructor.university/), Germany.
[  ](https://www.linkedin.com/in/konstantin-chaika/)
[Konstantin Chaika](https://www.linkedin.com/in/konstantin-chaika/) is an Educational Content Manager, Tutor at [JetBrains](https://www.jetbrains.com/), Czech Republic.
[  ](https://scholar.google.com/citations?user=EcmZ1fUAAAAJ&hl=en&oi=sra)
[Anton Filatov](https://scholar.google.com/citations?user=EcmZ1fUAAAAJ&hl=en&oi=sra) is currently affiliated with the [Saint Petersburg Electrotechnical University “LETI”](https://etu.ru/en/university/), Saint Petersburg, Russia.
[  ](https://ieeexplore.ieee.org/author/37086041250)
[Artyom Filatov](https://ieeexplore.ieee.org/author/37086041250 "Artyom Filatov") is currently affiliated with the [Saint Petersburg Electrotechnical University “LETI”](https://etu.ru/en/university/), Saint Petersburg, Russia.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** camera calibration, research
---
### [Visual localization using multi-camera multi-robot system](https://duckietown.com/multi-camera-multi-robot-visual-localization-system/)
**Published:** October 5, 2024
**Author:** Duckietown Admin
**Excerpt:** This research presents a framework for visual localization on rectangular maps with AprilTags, reducing setup allowing flexible camera placement in Duckietown.
**Content:**
##### General Information
- **Title**: Multi-camera multi-robot visual localization system
- **Authors**: Artur Morys Magiera, Marek Długosz, Paweł Skruch.
- **Institution**: [AGH University of Cracow](https://www.agh.edu.pl/en/), Poland
- **Citation**: A. M. Magiera, M. Długosz and P. Skruch, "Multi-camera multi-robot visual localization system," [ 2024 28th International Conference on Methods and Models in Automation and Robotics (MMAR) ](https://mmar.edu.pl/), Poland, 2024, pp. 375-380, doi: 10.1109/MMAR62187.2024.10680813.
[ Paper ](https://ieeexplore.ieee.org/abstract/document/10680813)
[ Institution ](https://www.agh.edu.pl/en/)
[ Authors ](#authors)
# Visual localization using multi-camera multi-robot system
**Visual robot localization** is a crucial problem in robotics: how to estimate the agents’ position using vision.
A common approach to solving it is through Simultaneous Localization and Mapping (**SLAM**) algorithms, using onboard sensors to map and estimate robot positions.
This work introduces a new algorithm for robot localization using **AprilTag** fiducial markers. It works on a rectangular map with four corner tags, requiring minimal configuration and offering flexibility in camera positions.
Unlike prior methods, this algorithm automatically stitches images from cameras, regardless of angle, and converts them into a **top-down view** for robot localization.
The approach promises **flexibility**, making adapting to dynamic camera setups easier without reconfiguration.
This solution offers automated robot localization with **minimal setup**, leveraging computer vision and AprilTags for more efficient mapping. The only constraint is the **rectangular shape of the map** and properly oriented **corner markers**, making it an ideal fit for scalable, adaptive robot environments.
Learn about robot autonomy, including perception, localization, and SLAM, starting from the link below!
[ Duckietown Educational Resources ](https://duckietown.com/educational-resources/#advanced-autonomy-approaches)
## Abstract
In the author’s words:
The article presents a general framework for detecting the boundaries of, stitching, adjusting perspective and finally localizing robot positions and azimuth angles for any rectangular map designated with AprilTag markers in the corners and possibly in the interior area.
At the same time, the focus of the researchers was to minimize the configuration required for the algorithm to operate – here limited to just the orientation and data of markers, dimensions of the map, markers and robots.
The location of cameras can be freely changed without the need to reconfigure anything or restart the program. This work has been tested on and turned out to be especially helpful for working with the Duckietown project.
## Highlights - Visual localization using multi-camera multi-robot system
Here is a visual tour of the work of the authors. For more details, check out the [full paper](#links "Post resources").
![A miniature Duckietown setup with small robot vehicles, roads, and AprilTag markers used for testing visual localization and autonomous navigation.]()
Figure 1. Duckietown Test Environment.
![A diagram showing common features (corner points) between two images captured by different cameras, used for stitching in the homography matrix algorithm.]()
Figure 2. Common Features Between Two Cameras.
![A diagram illustrating the stitching step masks, where overlapping areas of images are blended with equal weights, and disjoint areas are added with full opacity.]()
Figure 3. Stitching Step Masks for Image Blending.
![A stitched image with magenta points marking the map's corner boundaries, white cross marks for the corner markers' top-left corners, and red/blue crosses for inner markers.]()
Figure 4. Stitched Image with Map Boundaries and Marker Corners.
![A diagram showing corner reprojection to a top-down perspective with colorful lines connecting matching points, orange centroids of detections, and white labels for corner codes.]()
Figure 5. Corner Reprojection to Top-Down Perspective.
![A map showing robot positions marked by green crosses, with azimuth angles relative to north and coordinates represented as percentages of the map's dimensions.]()
Figure 6. Robot Position and Azimuth in Map Coordinates.
![A stitched image showing the extrapolation of one missing corner, where the algorithm estimates its position to create a complete view of the map.]()
Figure 7. Extrapolation of Missing Corner in Stitched View.
![A reprojection view showing the extrapolation of one missing corner, with the algorithm estimating the corner's position for a complete top-down perspective.]()
Figure 8. Extrapolation of Missing Corner in Reprojection View.
![The final result of missing corner extrapolation, showing a complete map with the estimated corner integrated smoothly into the overall environment.]()
Figure 9. Final Result of Missing Corner Extrapolation.
![A view showing common features in an image with no shared corners and three total corners, illustrating feature detection without overlapping corner points.]()
Figure 10. Common Features View with Zero Common Corners.
![A stitched image where no corners are shared but three total corners are present, showing how the algorithm combines images despite the absence of common corners.]()
Figure 11. Stitched View with Zero Common Corners.
![A reprojection view showing a scenario with no shared corners and three total corners, where the algorithm forms a complete top-down perspective despite the lack of common corners.]()
Figure 12. Reprojection View with Zero Common Corners.
![The result view showing a complete image with zero common corners and three total corners, illustrating the successful integration of data without overlapping features.]()
Figure 13. Result View with Zero Common Corners.
![A common features view showing zero common corners and three total corners, highlighting one common feature detected by the algorithm despite the lack of overlapping corners.]()
Figure 14. Common Features View with One Overlapping Feature.
![A stitched image showing zero common corners and three total corners, highlighting one common feature detected, demonstrating the algorithm's ability to combine images effectively.]()
Figure 15. Stitched View with One Common Feature.
![A reprojection view showing zero common corners and three total corners, emphasizing one common feature detected by the algorithm, illustrating effective data integration.]()
Figure 16. Reprojection View with One Common Feature.
![The result view showing zero common corners and three total corners, highlighting one common feature detected by the algorithm, demonstrating successful data integration.]()
Figure 17. Result View with One Common Feature.
## Conclusion - Visual localization using multi-camera multi-robot system
Here are the conclusions from the authors of this paper:
“The primary contribution and aim of this work is to provide a universal framework for stitching views of the same map from multiple cameras that can be freely moved and laid out around the map, with minimal required configuration.
The requirements for placement of codes are also loose: only the orientation with respect to the map frame is constrained and configuration of corner codes is required, as well as the lower limit of visible common markers on two images to be processed is 1, with no need for any corner markers to be present in both images at the same time.
The algorithms efficiency, however, depends on the quality of the homography matrices used in it, which implies that the more detections and corner detections, the better the result. It happens that the stitched / extrapolated coordinates may be off ’ground truth’ in some cases, or even stitching might fail, resulting in malformed output.
The authors provided experiments on two cameras, yet the algorithm may be run sequentially with images from more cameras. The algorithm may be improved in the future by applying more sophisticated methods of aggregating values of multiple detections of a given robot, such as a weighted combination of the position based on the quality of each detection.”
#### Project Authors
[  ](https://artus9033.vercel.app/)
[Artur Morys – Magiera](https://artus9033.vercel.app/) is a PhD candidate at [AGH University of Krakow](https://www.agh.edu.pl/en/), Poland.
[  ](https://drive.agh.edu.pl/team/002)
[Marek Długosz](https://drive.agh.edu.pl/team/002) is a graduate and faculty member of the Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering at the [AGH University of Science and Technology in Krakow](https://www.agh.edu.pl/en/), Poland.
[  ](https://www.drive.agh.edu.pl/team/001)
[Paweł Skruch](https://www.drive.agh.edu.pl/team/001) is a Professor of the [AGH University of Science and Technology](https://www.agh.edu.pl/en/), Poland.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Learn about Duckietown ](http://www.duckietown.com)
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** computer vision, localization, research
---
### [The 6th Annual Massrobotics Duckiedrone Academy](https://duckietown.com/duckiedrone-academy-massrobotics-summer-2024/)
**Published:** July 17, 2024
**Author:** Jacopo Tani
**Excerpt:** Duckietown partners with MassRobotics and Brown University to provide a week-long hands-on summer school in drone autonomy to high-school learners in Boston.
**Content:**
**Boston, MA, USA – Massrobotics**, July 2024: instructors and learners gather at MassRobotics in Boston to learn about drone autonomy.
##### Quick links
- [ MassRobotics 2024 Drone Academy ](https://www.massrobotics.org/massrobotics-recently-kicked-off-its-6th-drone-academy/)
- [ Duckiedrone (DD24) Manual ](https://docs.duckietown.com/daffy/opmanual-dd24/intro.html)
## The 6th Annual Drone Academy at MassRobotics
High school learners gathered at MassRobotics in Boston to learn about drone autonomy using the latest Duckiedrones, model DD24.
With the support of Brown University and Amazon Robotics, learners deep-dived for a week in the science and technology of autonomous flight.
Starting from a box of parts, the Duckietown DD24 drone and accompanying pedagogical materials enable a rich set of learning experiences for newcomers to autonomy, as well as for seasoned veterans.
Learners had the opportunity to practice soldering, electrical connections testing, software initialization for development and operations, actuator setup, sensor calibrations, low-level controller tuning, manual flight, and autonomous hovering.
This summer academy followed a similar experience at Howard University, Washington DC, that took place in June 2024.
The Duckiedrone is a DIY, Raspberry Pi-based drone designed to introduce learners to autonomous flight.
[ Get the Duckiedrone ](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24?variant=43227749023919)
[  ](https://get.duckietown.com/products/duckiebot-db21)
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
[ Request a quote for your class ](https://contact.duckietown.com/request-a-quote)
**Categories:** Events, News
**Tags:** dd24, duckiedrone
---
### [Duckiebot Intersection Navigation with DBSCAN](https://duckietown.com/dbscan-driven-intersection-navigation-for-duckiebots/)
**Published:** September 24, 2024
**Author:** Duckietown Admin
**Excerpt:** This project uses DBSCAN (Density-Based Algorithm for Discovering Clusters
in Large Spatial Databases with Noise) to improve Duckiebot intersection navigation.
**Content:**
# Duckiebot Intersection Navigation with DBSCAN
##### Project Resources
- **Objective**: Enable Duckiebots to navigate intersections safely, smoothly and efficiently.
- **Approach**: Using the DBSCAN (A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise) algorithm to detect stoplines and guide Duckiebots along precomputed optimal trajectories.
- **Authors**: Christian Leopoldseder, Matthias Wieland, Sebastian Seb Giles, Merlin Hosner, Amaury Camus.
[ Final Result ](#project-result)
[ Code ](https://github.com/duckietown-ethz/proj-lfi)
[ Authors ](#authors)
## Why intersection navigation using DBSCAN?
Navigating intersections is obviously important when driving in Duckietown. It is not as obvious that the mechanics of intersection navigation for autonomous vehicles are very different from those used for standard lane following. There typically is a finite state machine that transitions the agent behavior from one set of algorithms, appropriate for driving down the road, and a different set of algorithms, to actually solve the “intersections” problem.
The intersection problem in Duckietown has several steps:
1. Identifying the beginning of the intersection (identified with a horizontal red line on the road floor)
2. Stopping at the red line, before engaging the intersection
3. Identifying what kind of intersection it is (3-way or 4-way, according to the Duckietown appearance specifications at the time of writing)
4. Identifying the relative position of the Duckiebot at the intersection, hence the available routes forward
5. Choosing a route
6. Identifying when it is appropriate to engage the intersection to avoid potentially colliding with other Duckiebots (e.g., is there a centralized coordinator – a traffic light – or not?)
7. Engaging and navigating the intersection toward the chosen feasible route
8. Switching the state back to lane following.
Easier said than done, right?
For each of the points above different approaches could be used. This project focuses on improving the baseline solutions for points 2., and most importantly, 7. of the above.
The real challenge is the actual driving across the intersection (in a safe way, i.e., by “keeping your lane”), because the features that provide robust feedback control in the lane following pipeline are not present inside intersections. The baseline solution for this problem in Duckietown is open loop control, relying on the model of the Duckiebots and the Duckietown to magic-tune a few parameters and the curves just about right.
As all students of autonomy know, open-loop control is ideally perfect (when all models are known exactly), but it is practically pretty useless on its own, as “all models are wrong” \[learn why, e.g., in the [Modeling of a Differential Drive robot class](https://www.youtube.com/watch?v=XG4cODYVbJk&ab_channel=Duckietown "Modeling of a Differential Drive robot - Duckietown")\].
In this project, the authors seek to close the loop around intersection navigation, and chose to use an algorithm called “DBSCAN” (Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise) to do it.
DBSCAN ([Density-Based Spatial Clustering of Applications with Noise – wiki](https://en.wikipedia.org/wiki/DBSCAN "Density-Based Spatial Clustering of Applications with Noise - wiki")) is a clustering algorithm that groups data points based on density, identifying clusters of varying shapes and filtering out noise. It is used to find the red stop lines at intersections without needing predefined geometric priors (colors, shapes, or fixed positions). This allows to track meaningful visual features in intersections efficiently, localize with respect to them, and hence attempt to navigate along optimal precomputed trajectories depending on the chosen direction.
[ Learn about planning ](https://duckietown.com/educational-resources/#planning)
## Intersection navigation using DBSCAN: the challenges
Some of the challenges in this intersection navigation project are:
**Initial position uncertainty:** Duckiebot’s starting alignment at the stop line may vary, requiring the system to handle inconsistent initial conditions.
**Real-time feedback:** the current system lacks real-time feedback, relying on pre-configured instructions that cannot adjust for unexpected events, such as slippage of the wheels, inconsistencies between different Duckiebots, and misalignment of road tiles (non-compliant assembly).
**Processing speed:** previous closed-loop solution attempts used April tags and Kalman filters – with implementations that ended up being too slow: with low update rates and delays.
**Transition to lane following:** ensuring a smooth handover from intersection navigation to lane following requires precise control to avoid collisions and lane invasion.
## Project Highlights
Here is a visual tour of the output of the authors’ work. [Check out the GitHub repository ](#links "project resources")for more details!
![Image illustrating Duckiebot navigation options using DBSCAN at an intersection, showing paths for left, right, and straight turns.]()
Figure 1. Intersection Navigation Options for Duckiebots.
![Image showing three Duckiebot alignment scenarios at a stopline: poor alignment on the left and right, with optimal alignment in the center.]()
Figure 2. Duckiebot alignment relative to the initial stopline for a three-way intersection.
![Image of three Duckiebot camera views, each showing different field of view scenarios corresponding to poor, optimal, and poor alignments.]()
Figure 3. Duckiebot Camera's Field of View Based on Alignment.
![Image showing three navigation options for a Duckiebot at an intersection: turning left, turning right, and going straight.]()
Figure 4. Duckiebot Navigation Possibilities at an Intersection.
![Image showing the setup loop for testing Duckiebot intersection navigation performance.]()
Figure 5. Procedure for Performance Testing of Duckiebot Navigation.
![Image illustrating the performance testing loop for Duckiebot's navigation system.]()
Figure 6. Performance Testing Setup for Duckiebot Navigation.
![Image showing fixed intersection, stopline, and Duckiebot coordinate frames used in navigation.]()
Figure 7. Defined Coordinate Frames for Intersection Navigation.
![Comparison of original camera view, rectified camera view, and birdseye view images.]()
Figure 8. Camera Image Transformation: Original, Rectified, and Birdseye Views.
![Visualization of predicted stopline positions and clustering results with colored and white dots.]()
Figure 9. Clustering and Classification in Duckiebot Navigation.
![Visualization of pose estimates from clusters of red pixels with arrows indicating direction and a virtual lane representation.]()
Figure 10. Stopline Filtering and Pose Estimation in Duckiebot Navigation.
![Diagram illustrating the virtual lane in relation to the lane pose during a left turn by the Duckiebot.]()
Figure 11. Virtual Lane and Lane Pose for Left Turn Navigation.
![Diagrams depicting the handover condition and perspectives during the transition back to lane following for the Duckiebot.]()
Figure 12. Handover Conditions for Lane Following.
![Diagram illustrating how the intersection navigation system integrates with the existing Duckiebot architecture.]()
Figure 13. Integration of Intersection Navigation with the Existing System.
![Series of images showing the failed left turn of a Duckiebot due to a missing stopline, across five timesteps.]()
Figure 14. Sequence of Events Leading to a Failed Left Turn in a Three-Way Intersection.
![Sequence of images showing a Duckiebot overshooting a right turn but successfully recovering, across five timesteps.]()
Figure 15. Overshoot and Recovery During a Right Turn.
## Intersection Navigation using DBSCAN: Results
## Intersection Navigation using DBSCAN: Authors
[  ](https://www.linkedin.com/in/christian-leopoldseder/)
[Christian Leopoldseder](https://www.linkedin.com/in/christian-leopoldseder/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works as a Software Engineer at [Google](https://www.google.com/), Switzerland.
[  ](https://www.linkedin.com/in/matthias-wieland-7765a967)
[Matthias Wieland](https://www.linkedin.com/in/matthias-wieland-7765a967) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works as a Senior Consultant at [abaQon](http://www.abaqon.com/), Switzerland.
[  ](https://www.linkedin.com/in/sebgiles/)
[Sebastian Nicolas Giles](https://www.linkedin.com/in/sebgiles/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works as a Autonomous Driving Systems Engineer at [embotech](https://www.embotech.com/), Switzerland.
[  ](https://www.linkedin.com/in/merlin-hosner-206637128/)
[Merlin Hosner](https://www.linkedin.com/in/merlin-hosner-206637128/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works as a Process Development Engineer at [Climeworks](https://www.climeworks.com/), Switzerland. Merlin was a mentor on this project.
[  ](https://www.linkedin.com/in/amaury-camus-ethz/)
[Amaury Camus](https://www.linkedin.com/in/amaury-camus-ethz/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works as a Lead Robotics Engineer at [Hydromea](https://hydromea.com/), Switzerland. Amaury was a mentor on this project.
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Projects
**Tags:** project, student project
---
### [Obstacle Avoidance for Dynamic Navigation Using Obstavoid](https://duckietown.com/obstavoid-dynamic-obstacle-avoidance-in-duckietown/)
**Published:** September 7, 2024
**Author:** Duckietown Admin
**Excerpt:** The Obstavoid Algorithm enables obstacle avoidance in Duckietown in realtime, calculating optimal paths using a 3D grid for dynamic, collision-free navigation.
**Content:**
# Obstacle Avoidance for Dynamic Navigation Using Obstavoid
##### Project Resources
- **Objective**: Avoiding static and dynamic obstacles in a compliant Duckietown.
- **Approach**: Using 3D space-time grid, cost function modeling, and Dijkstra's algorithm.
- **Authors**: Alessandro Morra, Dominik Mannhart, Lionel Gulich, Victor Klemm..
[ Final Result ](#project-result)
[ Code ](https://github.com/duckietown/duckietown-mplan2)
[ Authors ](#authors)
## Why obstacle avoidance?
The importance of obstacle avoidance in self-driving is self-evident, whether the obstacle is a rubber duckie-pedestrian or another Duckiebot on the road.
In this project, authors deploy the **Obstavoid Algorithm** aiming to achieve:
- **Safety:** preventing collisions with obstacles and other Duckiebots, ensuring safe navigation in a dynamic environment.
- **Efficiency:** maintaining smooth movement by optimizing the trajectory, avoiding unnecessary stops or delays.
- **Real-world readiness:** preparing Duckietown for real-world scenarios where unexpected obstacles can appear, improving readiness.
- **Traffic management:** enabling better handling of complex traffic situations, such as maneuvering around blocked paths or navigating through crowded areas.
- **Autonomous operation:** It enhances the vehicle’s ability to operate autonomously, reducing the need for human intervention and improving overall reliability.
[ Learn about planning ](https://duckietown.com/educational-resources/#planning)
## Obstacle Avoidance: the challenges
Implementing obstacle avoidance in Duckietown introduces the following challenges:
- **Dynamic obstacle prediction:** accurately predicting the movement of dynamic obstacles, such as other Duckiebots, to ensure effective avoidance strategies and timely responses.
- **Computational complexity:** managing the computational load of the trajectory solver, in “real-time” scenarios with varying obstacle configurations, while ensuring efficient performance on limited computation.
- **Cost function design:** creating and fine-tuning a cost function that balances lane adherence, forward motion, and obstacle avoidance, while accommodating both static and dynamic elements in a complex environment.
- **Integration and testing:** ensuring integration of the Obstavoid Algorithm with the Duckietown simulation framework and testing its performance in various scenarios to address potential failures and refine its robustness.
The Obstavoid Algorithm addresses these challenges by employing a time-dependent cost grid and **Dijkstra’s algorithm** for optimal trajectory planning, allowing for “real-time” obstacle avoidance.
Read more about how the Dijkstra’s algorithm is used in this student project titled “[Goto-1: Planning with Dijkstra](https://duckietown.com/goto-1-planning-with-dijkstra/)“.
It dynamically calculates and adjusts trajectories based on predicted obstacle movements, ensuring navigation and integration with the simulation framework.
## Project Highlights
Here is the output of the authors’ work. [Check out the GitHub r epository ](#links "project resources")for more details!
![3D cost grid illustration depicting a weighted space-time grid, used for shortest path optimization in the Obstavoid Algorithm.]()
Figure 1. 3D Cost Grid Illustration for Obstavoid Algorithm.
![Graph showing a static cost function with a 6th-degree polynomial curve, representing lane following and forward motion in the Obstavoid Algorithm.]()
Figure 2. Static Cost Function for Lane Following and Forward Motion.
![Flowchart illustrating the software architecture of the Obstavoid Algorithm, detailing two main nodes: the trajectory creator node and the trajectory sampler node, and their communication with the simulation.]()
Figure 3. Software Architecture of the Obstavoid Algorithm.
![Graph showing the performance of the trajectory solver over 100 trajectory generations. The graph displays variability in solution times due to different obstacle configurations in the cost grid.]()
Figure 4. Performance Analysis of Trajectory Solver.
## Obstacle Avoidance: Results
## Obstacle Avoidance: Authors
[  ](https://www.linkedin.com/in/morraalessandro)
[Alessandro Morra](https://www.linkedin.com/in/morraalessandro/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently serves as the CEO & Co-Founder at [Ascento](https://www.ascento.ai/), Switzerland.
[  ](https://www.linkedin.com/in/dominikmannhart/)
[Dominik Mannhart](https://www.linkedin.com/in/dominikmannhart/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently serves as the Co-Founder at [Ascento](https://www.ascento.ai/), Switzerland.
[  ](https://www.linkedin.com/in/lionel-gulich/)
[Lionel Gulich](https://www.linkedin.com/in/lionel-gulich/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works as a Senior Robotics Software Engineer at [NVIDIA](http://www.nvidia.com/), Switzerland.
[  ](https://www.linkedin.com/in/vklemm/)
[Victor Klemm](https://www.linkedin.com/in/vklemm/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently is a PhD student at [Robotics Systems Lab](http://www.rsl.ethz.ch/), [ETH Zurich](https://ethz.ch/), Switzerland.
[  ](https://www.linkedin.com/in/lapandic)
[Dženan Lapandić](https://www.linkedin.com/in/lapandic) is a former Duckietown student and teaching assistant of the Autonomous Mobility on Demand class at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently is a PhD candidate at [KTH Royal Institute of Technology](http://www.kth.se/), Sweden.
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Projects
**Tags:** project, student project
---
### [Nurturing future engineers and leaders with Kevin Smith](https://duckietown.com/future-engineers-with-kevin-smith/)
**Published:** August 9, 2024
**Author:** Federico Tani
**Excerpt:** STEM instructor Kevin Smith shares his experience at MassRobotics: providing young learners with core skills to become future engineers and sector leaders.
**Content:**
# Nurturing future engineers and leaders with Kevin Smith
STEM instructor Kevin Smith shares his experience at MassRobotics: providing young learners with core skills to become future engineers and sector leaders.
**Boston, MA, USA, April 22nd 2024:** STEM Program Manager Kevin Smith shares his work at MassRobotics, a robotics startup incubator in Boston, MA, providing learning experiences to teach technological skills and inspire future engineers and industry leaders.
##### Quick links
- [ Kevin Smith on Linkedin ](https://www.linkedin.com/search/results/all/?heroEntityKey=urn%3Ali%3Afsd_profile%3AACoAAAXdQp0B_Z4Ukq-wBVUrSHkv0-Txc7S3CDk&keywords=Kevin%20Smith&origin=ENTITY_SEARCH_HOME_HISTORY&sid=gGk)
- [ MassRobotics ](https://www.massrobotics.org/)
- [ Summer Drone Academy ](https://www.massrobotics.org/stem/summer-drone-acadamy/)
- [ MassRobotics Jumspart Program ](https://www.massrobotics.org/stem/jumpstart-fellowship/)
## Teaching robotics to nurture future engineers and leaders
We talked with Kevin Smith from MassRobotics to learn more about his teaching activities and the programs he is involved in, such as the Jumpstart fellowship program and the Summer Duckiedrone Academy.
##### Good morning Kevin! May I ask you to start by introducing yourself?
Hi! My name is Kevin Smith. I have the pleasure of leading the STEM program here at MassRobotics. Our program encompasses creating STEM learning experiences ranging from two hours to six months long. The objective is to help students grow by leveraging our environment of 85-plus robotics startups. This ecosystem provides us with the opportunity to understand where technology is going in the next five to ten years, and we make sure that all of the learning experiences are ingrained with technical expertise.
##### Very nice. We know you use Duckietown for some of your teaching activities, when did you first run into it?
I discovered Duckietown for the first time at the Drone Academy that we were hosting here at MassRobotics, sponsored by Amazon Robotics.
We got a chance to build drones for the first time and really bring some high-level experiences to the students. It’s been a couple of years since I’ve been a part of the Drone Academy and we recently wrapped up the most recent edition where we got to kick off and try some [DD24](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24), the newest version of the Duckietown drones and we were so excited about it.
##### How do you use drones to inspire your students?
During the Drone Academy, we had the opportunity to teach students many dynamic skills within one week, building up a lot of core technical skills, but also using the experience as a point to have conversations with students to be able to open their minds to what happens beyond their High School environment.
So, for us, Duckiedrones are a huge technical resource to be able to teach technical skills, raise the level of rigor within the environment that we’re cultivating, and also help students think about what is beyond this phase that they’re in their life so they can become engineers in the future.
##### Could you tell us more about the Drone Academy?
The MassRobotics Drone Academy is a one-week summer camp targeted to high-school learners where we leverage our STEM space and partnerships with Amazon, Duckietown, and Brown University to create dynamic learning experiences.
Brown University worked on and cultivated some of the first drones which Duckietown has taken to extreme lengths to make sure that the product is top-tier quality. The camp is offered directly to students at no cost, ensuring that anyone who chooses to participate in these growth experiences can do so.
##### That is great, you make me want to join the next edition! Are there other programs at MassRobotics you would like to share with us?
At MassRobotics, we host a plethora of different STEM experiences. One of our premiere programs is the Jumpstart program where from January to May young ladies come every Saturday, or at least three out of four Saturdays every month, and work for about six to eight hours with industry experts on CAD \[Computer Assisted Design\] and CNC \[Computer Numerical Control\] machines. Essentially they learn engineering skills within a couple of months, and after that they are paid a thousand dollars for their commitment.
But I think the biggest part is the fact that they get to experience real-world internships, and actually some of our interns from the Jumsptart program had the honor of interning at Duckietown too, where they were able to assist in the development and student experience with these drones we’re speaking of!
##### It sounds awesome! Who is the target of the Jumsptart program?
Jumpstart is a program targeting young ladies in high school, we think juniors are the ideal age, because once they finish their January through May sessions and gain all the technical skills and soft skills, they will be holding up to walk right into an internship which is very difficult to line up for high school students as you can imagine.
It’ll also give them a lot to write on their applications going to college in terms of experience and exposure but also the skills that those colleges are looking for.
We’re trying to cultivate the next generation of leaders, while fostering the creation equitable environments to make sure that these young ladies who step into the internship realm feel comfortable and competent
> We're trying to cultivate the next generation of leaders, while fostering the creation of equitable environments to make sure that these young ladies who step into the internship realm feel comfortable and competent.
>
> Kevin Smith
##### Would you recommend Duckietown to colleagues and students?
Yes, I think Duckietown is a very cool and innovative platform that allows students to explore. It’s a platform to learn so many technical skills, but also to really start up a lot of conversations, and to digest what’s actually going on behind the scenes, how are these different components engaging and interacting with each other to create autonomous robots.
From the actual Duckietown \[the self-driving cars component of Duckietown\] I had the pleasure to see and work with, to the Duckiedrones, I believe they bring a lot for the students to explore and engage with.
### Learn more about Duckietown
Duckietown enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** News, People
**Tags:** AI-DO, duckiebot, robotics, student
---
### [The AI Driving Olympics at NIPS 2018](https://duckietown.com/the-ai-driving-olympics-at-nips-2018/)
**Published:** August 2, 2018
**Author:** Liam Paull
**Content:**
##### General Information
- The AI Driving Olympics at NIPS 2018
- Andrea Censi Liam Paull, Jacopo Tani, Julian Zilly, Thomas Ackermann, Oscar Beijbom, Berabi Berkai, Gianmarco Bernasconi, Anne Kirsten Bowser, Simon Bing, Pin-Wei David Chen, Yu-Chen Chen, Maxime Chevalier-Boisvert, Breandan Considine, Andrea Daniele, Justin De Castri, Maurilio Di Cicco, Manfred Diaz, Paul Aurel Diederichs, Florian Golemo, Ruslan Hristov, Lily Hsu, Yi-Wei Daniel Huang, Chen-Hao Peter Hung, Qing-Shan Jia, Julien Kindle, Dzenan Lapandic, Cheng-Lung Lu, Sunil Mallya, Bhairav Mehta, Aurel Neff, Eryk Nice, Yang-Hung Allen Ou, Abdelhakim Qbaich, Josefine Quack, Claudio Ruch, Adam Sigal, Niklas Stolz, Alejandro Unghia, Ben Weber, Sean Wilson, Zi-Xiang Xia, Timothius Victorio Yasin, Nivethan Yogarajah, Yoshua Bengio, Tao Zhang, Hsueh-Cheng Wang, Matthew Walter, Stefano Soatto, Magnus Egerstedt, Emilio Frazzoli
- ETH Zurich, Zurich, Switzerland
- The Springer Series on Challenges in Machine Learning, https://doi.org/10.1007/978-3-030-29135-8\_3
[ Paper ](https://www.duckietown.com/wp-content/uploads/2018/08/nips-2018-live-4-min.pdf)
[ Institution ](https://ethz.ch/en.html)
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Learn about Duckietown ](http://www.duckietown.com)
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** Blog, Manual
---
### [Successful failures: learning robotics with Prof. Bruegge](https://duckietown.com/prof-bruegge-interview-failing-successfully/)
**Published:** January 25, 2024
**Author:** Duckietown Admin
**Excerpt:** Prof. Bruegge and his colleagues about his experience using Duckietown and the new 4GB Duckiebots.
**Content:**
Prof. Bruegge and his colleagues about his experience using Duckietown and the new 4GB Duckiebots.
**Munich, April 3rd:** Prof. Bruegge (Brügge) studied computer science at the University of Hamburg and Carnegie Mellon University (CMU), where he also earned his doctorate. Appointed as Professor at the Technical University of Munich (TUM) in 1997, and also associate professor at CMU, Prof. Brügge was on the research committee of Deutsch Telekom and the Munich district. He has served on the board of directors of the Center for Digital Technology and Management, and acted as liaison professor for the Max Weber Foundation and the German National Academic Foundation (2000–2017).
##### Quick links
- [ Technical University of Munich ](https://www.professoren.tum.de/en/bruegge-bernd)
- [ Joint Advanced Student School (JASS) ](https://www.jass.school/jass2023)
- [ Google Scholar ](https://scholar.google.de/citations?user=io5O8OUAAAAJ&hl=de)
- [ Linkedin ](https://www.linkedin.com/in/bernd-br%C3%BCgge-7410101/?originalSubdomain=de)
# Failing successfully, an interview with Prof. Bruegge
## Prof. Bruegge and the JASS 2023 summer camp
Prof. Bruegge, Bernd, is an established computer scientist researcher and Professor, having (co)authored hundreds of peer reviewed publications with over 8800 citations at the time of this writing.
Emeritus Professor since 2021, Bernd regularly contributes to the organization and running of the [Joint Advanced Student School (JASS)](https://www.jass.school/ "Joint Advanced Student School (JASS)").
In this interview we capture his experience from the summer 2022 edition.
We are thrilled to have this chat with him!
##### Good morning! Could you introduce yourself?
My name is Bernd Bruegge. I was a Full Professor of Computer Science at the Technical University in Munich, I was at [Carnegie Mellon](https://www.cmu.edu/) before I joined the Technical University. I was for 20 years at the Computer Science Department at CMU in Pittsburgh, Pennsylvania.
##### What can you tell us about this JASS school event that took place in Cyprus?
The recent \[2023\] school was quite an adventure!
My colleague, Prof. [Kirill Krinkin](https://duckietown.com/community-spotlight-kirill-krinkin-stem-intensive-learning-approach/ "Duckietown community spotlight - Prof. Kirill Krinkin"), was in St. Petersburg. We had done all the JASS projects with Kirill and support activities by [JetBrains](https://www.jetbrains.com/ "JetBrains") in St. Petersburg. And as you know, it’s a total disaster there, there’s nothing left.
So we organized this school in Cyprus with 10 students from Munich and 10 from Cyprus and St. Petersburg. And once there, we built an advanced Duckiebot system, with focus on context sensitivity.
We looked not only at intersections and turns, but we also had times when the Duckiebots would have to slow down, or stop at repair sites where we had traffic lights. There were not only OCR codes, we used [Thread](https://en.wikipedia.org/wiki/Thread_(network_protocol)) and [Matter](https://en.wikipedia.org/wiki/Matter_(standard)). Are you familiar with Thread and Matter?
> I call this a "successful failure". The challenge in using advanced technology in the real world is that when you explore edge cases, unexpected situations may arise that one would not have considered by just simulating the same scenarios.
>
> Prof. Bernd Bruegge
##### Is this a new functionality you added to the system?
Yes, this is actually an add-on we made to Duckietown. It’s a new IP standard that allows to save energy much, much more than Bluetooth. It’s actually better by a factor of 10. We added Thread and Matter capabilities to our Duckietown environment.
##### That's fascinating! When did you first encounter Duckietown?
Well, I first heard of Duckietown a long time ago. As you know, I have been at Carnegie Mellon, which competed with MIT and Stanford. And when Kirill told me he would be a visiting professor at MIT, I asked: “What are you doing there?” And he said, “I’m using Duckietown!”
He was interested in robots and robotic technology and observed that this new technology is actually more like an environment and ecosystem than just a robot.
We then came up with a few ideas: first in St. Petersburg, we used a combination of drones and Duckiebots. Our scenario included two airports and a (duckie!) passenger. The drone had to pick up the duckie from one Duckiebot, and the students, divided into three teams, had to develop their own pickup mechanisms.
Each of them with different approaches. One used magnets, the other one used scoops, and so on. They had to then transport the duckie from airport A to airport B using indoor navigation GPS. There was our first really impressive demo.
**Credits**: Special thanks to Andreas Jung and Ruth Demmel, and the multimedia team supporting Prof. Bruegge.
##### Prof. Bruegge, do you feel you achieved your objectives?
I’d say we didn’t fulfill our technical objectives. The main challenge with Duckiebots was the third, omnidirectional, wheel, which had too much attrition. So, for instance, if we came to drive near a church, the idea would be to slow down the speed. But then we discovered it was difficult to control the Duckiebot at low speed, since when it comes down to a crawl, friction may cause it to stop. So operating at low speeds was not a practical possibility.
But I call this a “successful failure”. The challenge in using advanced technology in the real world is that when you explore edge cases, unexpected situations may arise that one would not have considered by just simulating the same scenarios.
I think the Duckiebot is good at lane-following and following traffic rules. When looking at context sensitivity though the challenges are trickier. So, e.g., we have a construction site with two [traffic lights](https://get.duckietown.com/products/traffic-light-dt22-tl "Duckietown Traffic Lights"), how to coordinate them so to minimize, e.g., traffic? If you have one Duckiebot coming from one side and one coming from the other, then one traffic light should go green, and the other one should be red.
But then we also have the idea that you have one traffic site that consists of multiple repair sites. In that situation the Duckiebots, lining up at the red traffic light, should have a green wave to go through each of the traffic lights until they exit the last traffic light.
So what we had in mind was coordination, making sure the Duckiebots were talking to each other. We used Python and the [Duckiebots with 4GB NVIDIA Jetson Nanos](https://get.duckietown.com/products/duckiebot-db21?variant=41543707099311 "Duckiebot with Jetson Nano 4GB (DB21-J4)"), and this happened during a five-day course.
##### Is there anything else you would like to add?
We plan to prepare a special room for students from primary schools, to show them the interplay of autonomous driving and context-sensitive or ubiquitous computing.
We would like to use the room in multiple ways. So we would like to have the Duckietown road sections, the tiles, interconnected in such a way that that allows us to switch between topographies basically in no time.
We have not been able to do this yet in an easy way, it’s always a scramble. Even if we use the same layout, the yellow tape is breaking or the white tapes are out of sync, and then we have to repair them.
So it would be great if we managed to keep the set up time for schools, especially for children (10 to 12 years), when doing exhibitions here, to under three minutes.
##### That's a great suggestion, thanks! Do you think Duckietown can be useful for younger learners too?
Yes, I think Duckietown can be very persuasive with young kids. They’re used to Lego or Fischer products, so when they meet the Duckiebot, that’s their first exposure to robots.
We actually have another project in mind, a series of Summer Schools. I’m a retired professor now, I don’t have to teach so I have time for Summer Schools! One will take place in the Dolomites, where the challenge will be a logistic one taking place in a factory. It will include robots moving packages around a warehouse.
### Learn more about Duckietown
Duckietown enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** News, People, Post categories
---
### [Monocular Visual Odometry for Duckiebot Navigation](https://duckietown.com/monocular-visual-odometry-for-duckiebot-navigation/)
**Published:** August 26, 2024
**Author:** Duckietown Admin
**Excerpt:** This project by Gianmarco Bernasconi, a former Duckietown student, provided an estimate of the Duckiebot's pose using a monocular visual odometry approach.
**Content:**
# Monocular Visual Odometry for Duckiebot Navigation
##### Project Resources
- **Objective**: Obtaining a "real-time" estimate of the pose of a Duckiebot traveling in Duckietown.
- **Approach**: Using monocular visual odometry.
- **Authors**: Gianmarco Bernasconi, Tomasz Firynowicz, Guillem Torrente, Yang Liu.
[ Final Result ](#project-result)
[ Code ](https://github.com/duckietown/duckietown-visualodo)
[ Authors ](#authors)
## Why Monocular Visual Odometry?
Monocular Visual Odometry (VO) falls under the “perception” block of the traditional robot autonomy architecture.
Perception in robot autonomy involves transforming sensor data into actionable information to accomplish a given task in the environment.
Perception is crucial because it allows robots to create a representation of themselves in teh environment they are operating within, which in turn enables the robot to navigate, avoid static or dynamic obstacles, forming the foundation for effective autonomy.
The function of monocular visual odometry is to estimate the robot’s pose over time by analyzing the sequence of images captured by a single camera.
VO in this project is implemented through the following steps:1. **Image acquisition:** the node receives images from the camera, which serve as the primary source of data for motion estimation.
2. **Feature extraction:** key features (points of interest) are extracted from the images using methods like ORB, SURF, or SIFT, which highlight salient details in the scene.
3. **Feature matching:** the extracted features from consecutive images are matched, identifying how certain points have moved from one image to the next.
4. **Outlier filtering:** erroneous or mismatched features are filtered out, improving the accuracy of the feature matches. In this project, histogram fitting to discard outliers is used.
5. **Rotation estimation:** the filtered feature matches are used to estimate the rotation of the Duckiebot, determining how the orientation has changed.
6. **Translation estimation:** simultaneously, the node estimates the translation, i.e., how much the Duckiebot has moved in space.
7. **Camera information and kinematics inputs:** additional information from the camera (e.g., intrinsic parameters) and kinematic data (e.g., velocity) help refine the translation and rotation estimations.
8. **Path and odometry outputs:** the final estimated motion is used to update the Duckiebot’s odometry (evolution of pose estimate over time) and the path it follows within the environment.
Monocular visual odometry is challenging, but provide low-cost, camera-based solution for real-time motion estimation in dynamic environments.
[ Write and test your own visual odometry algorithm ](https://duckietown.com/educational-resources/#education-materials-cv)
## Monocular Visual Odometry: the challenges
Implementing Monocular Visual Odometry involves processing images at runtime, presents challenges that effect performance. - Extracting and matching visual features from consecutive images is a fundamental task in monocular VO. This process can be hindered by factors such as low texture areas, motion blur, variations in lighting conditions and occlusions.
- Monocular VO systems face inherent scale ambiguity since a single camera cannot directly measure depth. The system must infer scale from visual features, which can be error-prone and less accurate in the absence of depth cues.
- Running VO algorithms requires significant computational resources, particularly when processing high-resolution images at a high frequency. The Raspberry Pi used in the Duckiebot has limited processing power and memory, which contrians the performance of the visual odometry pipeline (the newer Duckiebots, [DB21J](https://get.duckietown.com/products/duckiebot-db21) uses Jetson Nano for computation.)
- Monocular VO systems, as all [odometry systems relying on dead-recokning models](https://duckietown.com/educational-resources/#education-materials-modcon), are susceptible to long-term drift and divergence due to cumulative errors in feature tracking and pose estimation.
This project addresses visual odometry challenges by implementing robust feature extraction and matching algorithms (ORB by default) and optimizing parameters to handle dynamic environments and computational constraints. Moreover, it integrates visual odometry with existing Duckiebot autonomy pipeline, leveraging the finite state machine for accurate pose estimation and navigation.
## Project Highlights
Here is the output of the authors’ work. [Check out the GitHub repository ](#links "project resources")for more details!
![Image showing the estimated path of the Duckiebot during the visual odometry demo.]()
Figure 1. Path During Visual Odometry Demo.
![Image showing the process of removing outliers in visual odometry feature matching using histogram fitting.]()
Figure 2. Outlier Removal Using Histogram Fitting.
![Image illustrating the division of feature pairs into far and close regions for motion estimation in visual odometry.]()
Figure 3. Region Division for Motion Estimation.
![Block diagram showing the steps of the visual odometry pipeline for Duckiebot pose estimation.]()
Figure 4. Block Diagram of Visual Odometry Implementation.
## Monocular Visual Odometry: Results
## Monocular Visual Odometry: Authors
[  ](https://www.linkedin.com/in/gibernas/)
[Gianmarco Bernasconi ](https://www.linkedin.com/in/gibernas/)is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works as a Senior Research Engineer at [Motional](http://www.motional.com/), Singapore.
[  ](https://www.linkedin.com/in/tomasz-firynowicz)
[Tomasz Firynowicz](https://www.linkedin.com/in/tomasz-firynowicz) is a former Duckietown student and teaching assistant of the Autonomous Mobility on Demand class at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works as a Software Engineer at [Dentsply Sirona](http://www.dentsplysirona.com/), Switzerland. Tomasz was a mentor on this project.
[  ](https://www.linkedin.com/in/guillem-torrente-marti)
[Guillem Torrente Martí](https://www.linkedin.com/in/guillem-torrente-marti) is a former Duckietown student and teaching assistant of the Autonomous Mobility on Demand class at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works as a Robotics Engineer at [SonyAI](https://www.ai.sony/https://www.ai.sony/), Japan. Guillem was a mentor on this project.
[  ](https://www.linkedin.com/in/yang-liu-609b82134/)
Y[ang Liu](https://www.linkedin.com/in/yang-liu-609b82134/) is a former Duckietown student and teaching assistant of the Autonomous Mobility on Demand class at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently is a Doctoral Student at [EPFL](https://www.epfl.ch/en/), Switzerland. Yang was a mentor on this project
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Projects
**Tags:** project, student project
---
### [Duckiedrone: how to fly a Raspberry Pi-based autonomous quadcopter](https://duckietown.com/aerial-ros-community-meeting/)
**Published:** October 16, 2023
**Author:** Duckietown Admin
**Excerpt:** We discuss the latest Duckiedrone, a DIY Raspberry Pi-based autonomous quadcopter, architecture and Duckietown Sky roadmaps with the Aerial ROS Community.
**Content:**
**Boston, 20 October 2023**: Duckietown Sky and the Duckietown drone, a Raspberry Pi-based autonomous quadcopter, are discussed with the Aerial ROS community, a group of experts working to define the future of software architectures for quadcopters.
##### Quick links
- [ ROS Aerial community ](https://github.com/ros-aerial)
- [ Duckiedrone presentation ](https://hubs.ly/Q027dmD_0)
- [ Duckiedrone operation manual ](https://docs.duckietown.com/daffy/opmanual-duckiedrone/intro.html)
- [ Duckiedrone instructor manual ](https://docs.duckietown.com/daffy/course-intro-to-drones/intro.html)
## Learning robot autonomy by flying with Duckietown Sky
Following an invitation to the Aerial ROS workgroup community meeting, Duckietown staff was delighted to present the Duckietown Sky initiative, the current Duckiedrone design, a DIY Raspberry Pi-based autonomous quadcopter, and future plans for both hardware and courseware development.
[ Download the presentation ](https://hubs.ly/Q027dmD_0)
The goal of the ROS (Robotic Operating System) [aerial robotics working group](https://github.com/ros-aerial "ROS aerial robotics working group") is to gather drone enthusiasts within the ROS community and facilitate the sharing of ideas and discussion of issues regarding autonomous robotic platforms operating in the air.
Duckietown Sky, a National Science Foundation-funded educational effort in collaboration with Brown University started in 2019, is an integral component of the [Duckietown education vision](https://duckietown.com/mission/ "Duckietown Education Vision"), representing the commitment to fostering robot autonomy education in all its forms. Beyond self-driving cars (Duckiebots) and smart cities (Duckietowns), Duckietown highlights what is common despite the different applications of robot autonomy. From ground to sky, whether it drives, flies, or blinks, Duckietown is a platform to learn, explore and innovate when it comes to robot autonomy.
With the focus on quadcopters, Duckietown Sky offers MOOC-style learning experiences tailored for undergraduate and senior high school students. Flight is *exciting*!
The program’s design criteria revolves around achieving state-of-the-art autonomy ground-up using off-the-shelf components, with a Raspberry Pi as core computational unit, for its wide-spread applications and large community. Duckiedrones, now at the second hardware design iteration moving towards the third, aim to provide students with hands-on learning experiences covering from the basics, such as soldering, to pretty advanced algorithmic cornerstones of autonomy such an UKFs (Unscented Kalman Filters) and SLAM (Simultaneous Localization and Mapping).
Aspiring engineers should however be prepared for preliminary requirements like soldering, have access to a laptop or base station, and an internet connection for setting the working environment up.
[ Duckiedrone Operation Manual ](https://docs.duckietown.com/daffy/opmanual-duckiedrone/intro.html)
[ Duckiesky Instructor Manual ](https://docs.duckietown.com/daffy/course-intro-to-drones/intro.html)
## From a box of parts to a Raspberry Pi-based autonomous quadcopter
The Duckietown Sky experience is an exciting journey that begins with a simple box of parts and culminates in the creation of an autonomously flying drone. In the Duckietown spirit of democratizing access to the science and technology of autonomy through accessible platforms, the happy-yellow Duckiebox includes almost everything needed to get flying.
We encourage instructors, students and practitioners to check the development roadmaps for both our hardware design and courseware, outlined in our presentation, and reaching out without hesitation to provide comments or feedback!
Building on the extensive experience of the Duckietown team in massive open online courses (check out the [Self-Driving Cars with Duckietown MOOC](https://duckietown.com/mooc/ "Self-Driving Cars with Duckietown massive open online course"): the world’s first robot autonomy MOOC with hardware), we look to prepare a series of short online courses. These courses will be led by Professors from Brown, as well as other universities, and will provide an ever broader audience with the opportunity to explore the fascinating world of robot autonomy: from the science and technology to the tools and workflows, to real-world applications presented by industry and academic leaders.
[  ](https://get.duckietown.com/products/duckiedrone-dd21?variant=41159769522351)
[  ](https://get.duckietown.com/products/duckiedrone-dd21?variant=41159769522351)
Want to try the Duckietown Sky experience yourself, build a DYI Raspberry Pi-based autonomous quadcopter, or teach an aerial autonomy class at your high school or at university level? Follow the steps below to begin now.
[ 1. Get the Duckiedrone ](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24)
[ 2. Build the Duckiedrone ](https://docs.duckietown.com/daffy/opmanual-duckiedrone/intro.html)
[ 3. Fly the Duckiedrone ](https://docs.duckietown.com/daffy/course-intro-to-drones/intro.html)
### Learn more about Duckietown
Duckietown enables state-of-the-art robotics and AI learning experiences. It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
[ Request a quote for your class ](https://contact.duckietown.com/request-a-quote)
**Categories:** Blog, editor-choiche, education, Events, News
**Tags:** autonomousdrones, drones, engineeringeducation, learningautonomy, raspberrypi, robotics, technology
---
### [ProTip: Duckiebot Remote Connection](https://duckietown.com/protip-duckiebot-remote-connection/)
**Published:** September 5, 2024
**Author:** Duckietown Admin
**Excerpt:** Have you ever wanted to work from home but your Duckiebot is back at the lab? Learn how to access your Duckiebot from anywhere at any time.
**Content:**
# ProTip: Duckiebot Remote Connection
Have you ever wanted to work from home, but your robot is in the lab? Networks are notoriosly the trickyest aspect of robotics, and establishing a Duckiebot remote connection can be a real challenge.
The good news is, that as long as your Duckiebot has been left powered on, it is possible to establish a Duckiebot remote connection and operate the robot as if you were on the same network.
In this guide, we will show how to access your Duckiebot from anywhere in the world using [ZeroTier](https://www.zerotier.com/).
[ Duckiebot remote control setup instructions ](https://docs.duckietown.com/daffy/opmanual-duckiebot/protips/duckiebot-remote-connection.html)
## ProTips
Knowing the science does not necessarily mean being practical with the tips and tricks of the roboticist job. “ProTips” are professional tips discussing (apparently) “small details” of the everyday life of a roboticist.
We collect these tips to create a guideline for “best practices”, whether for saving time, reducing mistakes, or getting better performances from our robots. The objective is to share professional knowledge in an accessible way, to make the life of every roboticist easier!
If you would like to contribute a ProTip, [reach out](https://duckietown.com/contact/ "Duckietown contact information").
## About Duckietown
Duckietown is a platform that streamlines teaching, learning, and doing research on robot autonomy by offering hardware, software, curricula, technical documentation, and an international community for learners.
Check out the links below to learn more about Duckietown and start your learning or teaching adventure.
[ Get Started ](https://www.duckietown.com/guides)
[ Pricing ](https://duckietown.com/pricing/)
**Categories:** Blog, Projects
**Tags:** blog, ProTip
---
### [Towards Autonomous Driving with Small-Scale Cars: A Survey of Recent Development](https://duckietown.com/towards-autonomous-driving-with-small-scale-cars-a-survey-of-recent-development/)
**Published:** June 30, 2024
**Author:** Duckietown Admin
**Excerpt:** This survey reviews small-scale car platforms and Autonomous Driving as cost-effective educational and research tools, advancing robot autonomy.
**Content:**
##### General Information
- Towards Autonomous Driving with Small-Scale Cars: A Survey of Recent Development
- Dianzhao Li, Paul Auerbach, Ostap Okhrin
- Technische Universität Dresden, Germany
[ arXiv ](https://arxiv.org/abs/2404.06229)
[ University ](https://tu-dresden.de/?set_language=en)
[ Authors ](#authors)
# Towards Autonomous Driving with Small-Scale Cars: A Survey of Recent Development
“**Towards Autonomous Driving with Small-Scale Cars: A Survey of Recent Development** by Dianzhao Li, Paul Auerbach, and Ostap Okhrin is a review that highlights the rapid development of the industry and the important contributions of small-scale car platforms to robot autonomy research.
This survey is a valuable resource for anyone looking to get their bearings in the landscape of autonomous driving research.
We are glad see Duckietown – not only included on the list – but identified as one of the platforms that started a marked increase in the trend of yearly published papers.
The mission of Duckietown, since we started at as a class at [MIT](https://www.mit.edu/), is to *democratize access to the science and technology of robot autonomy*. Part of how we intended to achieve this mission was to streamline the way autonomous behaviors for non-trivial robots were developed, tested and deployed in the real world.
From 2018-2021 we ran several editions of the [AI Driving Olympics](https://duckietown.com/research/ai-driving-olympics/ "Duckietown AI Driving Olympics") (AI-DO): an international competition to benchmark the state of the art of embodied AI for safety-critical applications. It was a great experience – not only because it led to the development of the [Challenges infrastructure](https://challenges.duckietown.org/v4/ "Duckietown Challenges infrastructure"), the Autolab infrastructure, and many agent baselines that catalyze further developments that are now available to the broader community, but even because it was the first time physical robots were brought the world’s leading scientific conference in Machine Learning ([NeurIPS](https://neurips.cc/ "Neural Information Processing Systems Conference"): the Neural Information Processing Systems conference – known as NIPS the first time AI-DO was launched).
All this infrastructure development and testing might have been instrumental in making R&D in autonomous mobile robotics more efficient. Practitioners in the field know-how doing R&D is particularly difficult because final outcomes are the result of the tuple (robot) x (environment) x (task) – so not standardizing everything other than the specific feature under development (i.e., not following the *ceteris paribus* principle) often leads to apples and pair comparisons, i.e., bad science, which hampers the overall progress of the field.
We are happy to see Duckietown recognized as a contributor to facilitating the making of good science in the field. We beleive that even better and more science will come in the next years, as the students being educated with the Duckietown system start their professional journeys in academia or the workforce.
We are excited to see what the future of robot autonomy will look like, and we will continue doing our best by providing tools, workflows, and comprehensive resources to facilitate the professional development of the next generations of scientists, engineers, and practicioners in the field!
To learn more about Duckietown teaching resources follow the link below.
> Starting around 2016, with the introduction of Duckietown, BARC, and Autorally, there was a significant increase in research papers.
>
> Dianzhao Li, Paul Auerbach, Ostap Okhrin
[ Duckietown Educational Resources ](https://duckietown.com/educational-resources/#machine-learning)
## Abstract
We report the abstract of the authors’ work:
“While engaging with the unfolding revolution in autonomous driving, a challenge presents itself, how can we effectively raise awareness within society about this transformative trend? While full-scale autonomous driving vehicles often come with a hefty price tag, the emergence of small-scale car platforms offers a compelling alternative.
These platforms not only serve as valuable educational tools for the broader public and young generations but also function as robust research platforms, contributing significantly to the ongoing advancements in autonomous driving technology.
This survey outlines various small-scale car platforms, categorizing them and detailing the research advancements accomplished through their usage. The conclusion provides proposals for promising future directions in the field.”
## Towards Autonomous Driving with Small-Scale Cars: A Survey of Recent Development
Here is a visual tour of the work. For more details, check out the [full paper](#links "Post resources").
![Based on published studies each year on Google Scholar with the search terms: robot car, the research on small-scale cars has seen substantial growth in the number of papers over the years. In the early 2000s, projects like s-bot, e-puck, and TurtleBot emerged. Starting around 2016, with the introduction of Duckietown, BARC, and Autorally, there was a significant increase in research papers. This trend continued with the development of projects like DeepRacer, Donkeycar, and F1TENTH. More computationally advanced small-scale cars have been introduced in recent years, such as ART/ATK and XTENTH-CAR.]()
Fig. 1-A. Illustration of the development and current states of small-scale car platforms, each depicted platform image sourced from its respective paper or website.
![amples of small-scale car platforms, categorized into educational platforms and research platforms, including multiple vehicle setups such as ORAC and UDSSC]()
Fig 1-B. Samples of small-scale car platforms, categorized into educational platforms and research platforms, including multiple vehicle setups such as ORAC and UDSSC
![Comparison of two pipelines for the autonomous driving system. An end-to-end system maps raw sensor inputs directly into control commands, whereas a modular system includes multiple subsystems to process the sensor inputs sequentially and output control commands.]()
Fig. 2. Comparison of two pipelines for the autonomous driving system.
![Research in the literature concerning small-scale cars is classified into various benchmark tasks in this survey, ranging from localization and mapping, path planning and following, lane keeping to racing, and cooperative driving.whereas a modular system includes multiple subsystems to process the sensor inputs sequentially and output control commands.whereas a modular system includes multiple subsystems to process the sensor inputs sequentially and output control commands.]()
Fig. 3. Research in the literature concerning small-scale cars is classified into various benchmark tasks in this survey, ranging from localization and mapping, path planning and following, lane keeping to racing, and cooperative driving.whereas a modular system includes multiple subsystems to process the sensor inputs sequentially and output control commands.
![Overall perception and control pipeline. The perception module receives onboard sensor data and produces a predicted occupancy map using a U-Net style generative neural network. The control algorithm receives the robot state, predicted occupancy map, and goal point and generates collision-free trajectories with RRT]()
Fig. 4. Overall perception and control pipeline. The perception module receives onboard sensor data and produces a predicted occupancy map using a U-Net style generative neural network. The control algorithm receives the robot state, predicted occupancy map, and goal point and generates collision-free trajectories with RRT
![The probabilistic dynamics model is trained using a multistep loss that considers how uncertainty propagates. The trajectory tracking controller is used to predict a distribution of closed-loop trajectories. The divergence constrained optimizer is used to find a closed- loop trajectory with low divergence for MuSHR cars]()
Fig. 5. The probabilistic dynamics model is trained using a multistep loss that considers how uncertainty propagates. The trajectory tracking controller is used to predict a distribution of closed-loop trajectories. The divergence constrained optimizer is used to find a closed- loop trajectory with low divergence for MuSHR cars
![Architecture of the proposed adaptive trajectory tracking control system]()
Fig. 6. Architecture of the proposed adaptive trajectory tracking control system
![Different outputs of the multi-step image processing pipeline start from the image input from the camera, proceed to detect road markings according to the color with HSV thresholding, detect marking edges with a canny filter, and finalize with detection marking used to estimate the lateral displacement and angle offset.]()
Fig. 7. Image Processing Pipeline for Road Marking Detection and Lateral Displacement Estimation
![Image patches are encoded using a ViT and predict a class label for each patch. Then, the coarse segmentation output is used for a potential-field based controller.]()
Fig. 8. Image patches are encoded using a ViT and predict a class label for each patch. Then, the coarse segmentation output is used for a potential-field based controller.
![Framework proposes the perception module leverages camera images to produce impact attributes regarding the environment, then the DRL control module utilizes the information to control the agent perform car following behavior.]()
Fig. 9. Framework for Car Following Behavior Using Perception and DRL Control Modules
![The cost map of the track in front of the vehicle is predicted with CNN.]()
Fig. 10. The cost map of the track in front of the vehicle is predicted with CNN.
![End-to-end IL system used for an Autorally racing car.]()
Fig. 11. End-to-end IL system used for an Autorally racing car.
![Overall schematics of the multi-vehicle, mixed reality RL approach. Both virtual and real DeepRacer vehicles exist within the simulation that manages the physics of the virtual cars and emulates collisions in mixed reality.]()
Fig. 12. Overall schematics of the multi-vehicle, mixed reality RL approach. Both virtual and real DeepRacer vehicles exist within the simulation that manages the physics of the virtual cars and emulates collisions in mixed reality.
![Diagram of trajectory planner with C-MOBIL and C-IDM. The ego vehicle plans a trajectory and velocity profile based on its state and the actual state of the neighboring vehicles]()
Fig. 13. Diagram of trajectory planner with C-MOBIL and C-IDM. The ego vehicle plans a trajectory and velocity profile based on its state and the actual state of the neighboring vehicles
## Summary and conclusion
Here is what the authors learned from this survey:
“In this paper, we offer an overview of the current state-of-the- art developments in small-scale autonomous cars. Through a detailed exploration of both past and ongoing research in this domain, we illuminate the promising trajectory for the advancement of autonomous driving technology with small-scale cars. We initially enumerate the presently predominant small-scale car platforms widely employed in academic and educational domains and present the configuration specifics of each platform. Similar to their full-size counterparts, the deployment of hyper-realistic simulation environments is imperative for training, validating, and testing autonomous systems before real-world implementation. To this end, we show the commonly employed universal simulators and platform-specific simulators.
Furthermore, we provide a detailed summary and categorization of tasks accomplished by small-scale cars, encompassing localization and mapping, path planning and following, lane-keeping, car following, overtaking, racing, obstacle avoidance, and more. Within each benchmarked task, we classify the literature into distinct categories: end-toend systems versus modular systems and traditional methods 20 versus ML-based methods. This classification facilitates a nuanced understanding of the diverse approaches adopted in the field. The collective achievements of small-scale cars are thus showcased through this systematic categorization. Since this paper aims to provide a holistic review and guide, we also outline the commonly utilized in various well-known platforms. This information serves as a valuable resource, enabling readers to leverage our survey as a guide for constructing their own platforms or making informed decisions when considering commercial options within the community.
We additionally present future trends concerning small-scale car platforms, focusing on different primary aspects. Firstly, enhancing accessibility across a broad spectrum of enthusiasts: from elementary students and colleagues to researchers, demands the implementation of a comprehensive learning pipeline with diverse entry levels for the platform. Next, to complete the whole ecosystem of the platform, a powerful car body, varying weather conditions, and communications issues should be addressed in a smart city setup. These trends are anticipated to shape the trajectory of the field, contributing significantly to advancements in real-world autonomous driving research.
While we have aimed to achieve maximum comprehensiveness, the expansive nature of this topic makes it challenging to encompass all noteworthy works. Nonetheless, by illustrating the current state of small-scale cars, we hope to offer a distinctive perspective to the community, which would generate more discussions and ideas leading to a brighter future of autonomous driving with small-scale cars.”
## Project Authors
[Dianzhao Li](https://ieeexplore.ieee.org/author/388655525177583) is a research assistant at the [Technische Universität Dresden](https://tu-dresden.de/), Dresden, Germany.
[Paul Auerbach](https://www.pauerbach.de/) is with [Barkhausen Institut gGmbH](https://www.barkhauseninstitut.org/en/), Dresden, Germany
[Ostap Okhrin](https://ieeexplore.ieee.org/author/484634543471489) is Chair of Statistics and Econometrics at the Institute of Economics and Transport, School of Transportation, [Technische Universitat Dresden](https://tu-dresden.de/) in Germany.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Learn about Duckietown ](http://www.duckietown.com)
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**End-to-end Deep RL (DRL) systems:** in autonomous driving environments that rely on visual input for vehicle control face potential security risks, including:
- State Adversarial Perturbations: Subtle alterations to visual input that mislead the DRL agent, causing incorrect decision-making.
- Reward Tampering: Manipulation of the reward signal to misguide the learning process, leading the agent to adopt unsafe or inefficient policies.
These vulnerabilities can compromise the safety and reliability of self-driving vehicles.
**Categories:** editor-choiche, Research
**Tags:** research, survey
---
### [YOLO-based Robust Object Detection in Duckietown](https://duckietown.com/yolo-based-robust-object-detection/)
**Published:** July 20, 2024
**Author:** Duckietown Admin
**Excerpt:** This project implements robust object detection in Duckietown for Duckiebots under varying lighting conditions and object clutter using a YOLO-based NN.
**Content:**
# YOLO-based Robust Object Detection in Duckietown
##### Project Resources
- **Objective**: Implementing Advanced and Robust Object Detection for Duckietown.
- **Approach**: Enhancing object detection robustness in Duckietown by utilizing a YOLO-based neural network trained on augmented datasets, focusing on safety and performance across varied lighting conditions.
- **Authors**: Maximilian Stölzle and Stefan Lionar
[ Final Result ](#project-result)
[ Code ](https://github.com/duckietown-ethz/proj-lfivop-ml)
[ Authors ](#authors)
## Why Robust Object Detection?
Object detection is the ability of a robot to identify a feature in its surroundings that might influence its actions. For example, if an object is laid on the road it might represent an obstacle, i.e., a region of space that the Duckiebot cannot occupy. Robust object detection becomes particularly important when operating in dynamic environmental conditions.
Obstacles can be of various, shape or color and they can be detected through different sensing modalities, for example, through vision or lidar scanning.
In this project, students use a purely vision-based approach for obstacle detection. Using vision is very tricky because small nuisances such as in-class variations (think of many different type of duckies) or environmental lighting conditions will dramatically affect the outcome.
Robust object detection refers to the ability of a system to detect objects in a broad spectrum of operating conditions, and to do so reliably.
Detecting object in Duckietown is therefore important to avoid static and moving obstacles, detect traffic signs and otherwise guarantee safe driving.
## Robust Object Detection: the challenges
Some of the key challenges associated with vision-based object detection are the following:
**Robustness across variable lighting conditions:** Ensuring accurate object detection under diverse lighting is complex due to changes in object appearance ([check out why in our computer vision classes](https://duckietown.com/educational-resources/#education-materials-cv)). The model must handle different lighting scenarios effectively.
**Balancing robustness and performance:** There’s a trade-off between robustness to lighting variations and achieving high accuracy in standard operating conditions. Prioritizing one may affect the other.
**Integration and real-time performance:** Integrating the trained neural network (NN) model into the Duckiebot’s system is required for real-time operation, avoid lags associated with transport of images across networks. The model’s complexity therefore must align with the computational resources available. This project was executed on [DB19 model Duckiebots](https://docs.duckietown.com/daffy/opmanual-duckiebot/preliminaries_hardware/duckiebot_configurations/index.html "Duckiebot configurations"), equipped with Raspberry Pi 3B+ and a Coral board.
**Data quality and generalization:** Ensuring the model generalizes well despite potential biases in the training dataset and transfer learning challenges is crucial. Proper dataset curation and validation are essential.
## Project Highlights
Here is the output of their work. [Check out the github repository ](#links "project resources")for more details!
![Figure 1 shows the appearance of the Duckiebot with Google Coral USB Accelerator to enable real-time and online inference for the Robust Object Detection.]()
Figure 1. Duckiebot with Google Coral USB Accelerator.
![Inference results sample showing annotated objects detected by the fully safety-weighted model in Duckietown.]()
Figure 2. Sample of inference results of our fully safety-weighted model.
![Proposed locational weights for classification loss shown on an overlayed sample image (a) and a heatmap (b).]()
Figure 3. Proposed locational weights for classification loss.
![Schematic of the object detection node showing the workflow between ROS tasks (Python2) and inference tasks (Python3) using separate scripts and temporary files for data exchange.]()
Figure 4. Schematic of object detection node.
![Sample inference on validation set showing predictions by Extra Augmentation, Fully Safety-weighted, and Vanilla models.]()
Figure 5. Sample inference on validation set.
![Sample inferences on new images under normal and low lights, showing detection performance of Vanilla, Extra Augmentation, and Fully Safety-weighted models.]()
Figure 6. Sample inferences on new images taken under normal and low lights.
![Sample inferences under normal and highly illuminated backgrounds, comparing the performance of Vanilla, Extra Augmentation, and Fully Safety-weighted models.]()
Figure 7. Sample inferences on new images taken in under normally and highly illuminated backgrounds.
## Robust Obstacle Detection: Results
## Robust Object Detection: Authors
[  ](https://www.linkedin.com/in/maximilian-stoelzle-501a0657/?originalSubdomain=nl)
[Maximilian Stölzle](https://www.linkedin.com/in/nikolaj-witting-a2395a131/) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works at [MIT](https://www.mit.edu/) as a Visiting Researcher.
[  ](https://www.linkedin.com/in/stefan-putra-lionar/?originalSubdomain=sg)
[Stefan Lionar](https://www.linkedin.com/in/stefan-putra-lionar/?originalSubdomain=sg) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), currently an Industrial PhD student at[ Sea AI Lab (SAIL)](https://sail.sea.com/), Singapore.
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Projects
**Tags:** project, student project
---
### [Goto-1: Planning with Dijkstra](https://duckietown.com/goto-1-planning-with-dijkstra/)
**Published:** August 5, 2024
**Author:** Duckietown Admin
**Excerpt:** This project enhances Duckiebot planning capabilities for autonomous navigation in Duckietowns using the Dijkstra algorithm.
**Content:**
# Goto-1: Planning with Dijkstra
##### Project Resources
- **Objective**: Having a Duckiebot drive autonomously from a starting position to any compliant ending position in Duckietown., and stopping there.
- **Approach**: Implementing Dijkstra's algorithm for route planning and achieving global localization leveraging the Duckietown appearance specifications and indefinite navigation framework.
- **Authors**: Johannes Boghaert
[ Final Result ](#project-result)
[ Code ](https://github.com/duckietown-ethz/proj-goto-1)
[ Authors ](#authors)
## Why planning with Dijkstra?
Planning is one of the three main components, or “blocks”, in a traditional robotics architecture for autonomy: “to see, to plan, to act” (perception, planning, and control).
The function of the planning “block” is to provide the autonomous decision-making part of the robots’ mind, i.e., the controller, with a reference path to follow.
In the context of Duckietown, planning in applied at different hierarchical levels, from lane following to city navigation.
This project aimed to build upon the vision-based lane following pipeline, introducing a deterministic planning algorithm to allow one Duckiebot to go to from any location (or tile) on a compliant Duckietown map to a specific target tile (hence the name: *Goto-1*).
Dijkstra algorithm is a graph-based methodology to determine, in a computationally efficient manner, the shortest path between two nodes in the graph.
[ Write and test your own Dijkstra algorithm ](https://duckietown.com/educational-resources/#planning)
## Autonomous Navigation: the challenges
The new planning capailities of Duckiebots enable autonomous navigation building on pre-existing functionalities, such as “lane following”, “intersection detection and identification”, and “intersection navigation” (we are operating in a scenario with only one agent on the map, so coordination and obstacle avoidance are not central to this progect).
Lane following in Duckietown is mainly vision-based, and as such suffers from the typical challenges of vision in robotics: motion blur, occlusions, sensitivity to environmental lighting conditions and “slow” sampling.
Intersection detection in Duckietown relies on the identification of the red lines on the road layer. Identification of the type of intersection, and relative location of the Duckiebot with respect to it, is instead achieved through the detection and interpretation of fiducial markers, appropriately specified and located on the map. In the case of Duckietown, April Tags (ATs) are used. Each AT, in addition to providing the necessary information regarding the type of intersection (3- or 4-way) and the position of the Duckiebot with respect to the intersection, is mapped to a unique ID in the Duckietown traffic sign database.
These traffic signs IDs can be used to unamiguosly define the graph of the city roads. Based on this, and leveraging the lane following pipeline state estimator, it is possible to estimate the location (with tile accuracy) of the Duckiebot with respect to a global map reference frame, hence providing the agent sufficient information to know when to stop.
After stopping at an intersection, detecting and identifying it, Duckiebots are ready to choose which direction to go next. This is where the Dijkstra planning algorithm comes into play. After the planner communicates the desired turn to take, the Duckiebot drives through the intersection, before switchng back to lane following behavior after completing the crossing. In Duckietown, we refer to the combined operation of these states as “indefinite navigation”.
Switching between different “states” of the robot mind (lane following, intersection detection and identification, intersection navigation, and then back to lane following) requires the careful design and implementation of a “finite state machine” which, triggered by specific events, allows for the Duckiebot to transition between these states.
Integrating a new package within the existing indefinite navigation framework can cause inconsistencies and undefined behaviors, including unreliable AT detection, lane following difficulties, and inconsistent intersection navigation.
Performance evaluation of the GOTO-1 project involved testing three implementations with ten trials each, revealing variability in success rates.
## Project Highlights
Here is the output of their work. [Check out the GitHub repository ](#links "project resources")for more details!
![Diagram showing a Duckietown layout with three key sub-problems highlighted: Localization/Planning (A), Navigation, and Arrival/Stopping (B). The starting position of the Duckiebot is marked as 'A,' and the final destination is marked as 'B.']()
Figure 1. Visualization of the three key sub-problems in the GOTO-1 project within a Duckietown configuration.
![Flowchart of the GOTO-1 pipeline. Yellow blocks represent components from the existing indefinite navigation framework, orange blocks represent custom GOTO-1 modules, and unmarked inputs are either provided or self-determined.]()
Figure 2. Overview of the GOTO-1 pipeline, showing the integration of custom blocks and existing framework components.
![Three images showing the transformation of a Duckietown map into two input matrices used for path planning: a graphical map definition, a cost matrix, and an input matrix representing allowable turn commands.]()
Figure 3. Transformation of the Duckietown map configuration into input matrices for path planning.
![Image showing the cropping of a mask and summing pixel values to detect midline stripes.]()
Figure 4. Processing image data to track midline stripes by cropping and summing pixel values.
![Charts evaluating the efficiency of the GOTO-1 system and the accuracy of apriltag detections.]()
Figure 5. Assessment of GOTO-1's efficiency and the accuracy of apriltag (AT) detections.
![Terminal window showing the initialization of GOTO-1 with a detailed listing of input and tuning parameters. Subsequent localization and path planning actions are logged after lane following is started.]()
Figure 6. Initialization of GOTO-1 from the terminal with all parameters listed, followed by localization, path planning, and lane following actions logged.
![Terminal window displaying logs of navigation, state estimation for the last mile problem, and the shutdown sequence of the Duckiebot upon reaching its destination.]()
Figure 7. Logging of navigation, state estimation for the last mile problem, and shutdown of the Duckiebot upon arrival.
## Autonomous Navigation: Results
## Autonomous Navigation: Authors
[  ](https://www.linkedin.com/in/johannes-boghaert)
[Johannes Boghaert](https://www.linkedin.com/in/johannes-boghaert) is a former Duckietown student of class Autonomous Mobility on Demand at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently serves as the CEO of [Superlab Suisse](http://www.superlabsuisse.com/), Switzerland.
[  ](https://www.linkedin.com/in/merlin-hosner-206637128)
[Merlin Hosner](https://www.linkedin.com/in/merlin-hosner-206637128) is a former Duckietown student and teaching assistant of the Autonomous Mobility on Demand class at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently works as Process Development Engineer at [Climeworks](http://climeworks.com/), Switzerland. Merlin was a mentor on this project.
[  ](https://www.linkedin.com/in/gioele-zardini-9459a1155/)
[Gioele Zardini](https://www.linkedin.com/in/gioele-zardini-9459a1155/) is a former Duckietown student and teaching assistant of the Autonomous Mobility on Demand class at [ETH Zurich](https://www.linkedin.com/school/eth-zurich/), and currently is an Assistant Professor at [MIT](http://web.mit.edu/). Merlin was a mentor on this project.
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Projects
**Tags:** project, student project
---
### [Enhancing Visual Domain Randomization for Sim2Real Transfer](https://duckietown.com/enhancing-visual-domain-randomization-for-sim2real-transfer/)
**Published:** July 27, 2024
**Author:** Duckietown Admin
**Excerpt:** Domain randomization helps sim-to-real-transfer. This work focuses on visual domain randomization, adding real images to synthetic ones from Duckietown Gym.
**Content:**
##### General Information
- Enhancing Visual Domain Randomization with Real Images for Sim-to-Real Transfer
- András Béres and Bálint Gyires-Tóth
- Budapest University of Technology and Economics, Hungary
- Béres, András and Gyires-Tóth, Bálint (2023) Enhancing Visual Domain Randomization with Real Images for Sim-to-Real Transfer. INFOCOMMUNICATIONS JOURNAL, 15 (1). pp. 15-25. ISSN 2061-2079
[ Paper ](https://real.mtak.hu/165177/)
[ Institution ](https://www.bme.hu/en)
[ Authors ](#authors)
# Enhancing Visual Domain Randomization with Real Images for Sim-to-Real Transfer
One of the classical objections made to machine learning approaches to embeddded autonomy (i.e., to create agents that are deployed on real, physical, robots) is that training requires data, data requires experiement, and experiment are “expensive” (time, money, etc.).
The natural counter argument to this is to use simulation to create the training data, because simulations are much less expensive than real world experiment; they can be ran continuously, with accellerated time, don’t require supervision, nobody gets tired, etc.
But, as the experienced roboticist knows, “simulations are doomed to succeed”. This phrase encapsulates the notion that simulations do not contain the same wealth if information as the real world, because they are programmed to be what the programmer wants them to be useful for – they do not capture the complexity of the real world. Eventually things will “work” in simulation, but does that mean they will “work” in the real-world, too?
As Carl Sagan once said: “If you wish to make an applie pie from scratch, you must first reinvent the universe”.
Domain randomization is an approach to mitigate the limitations of simulations. Instead of training an agent on one set of parameters defining the simulation, many simulations are instead ran, with different values of this parameters. E.g., in the context of a driving simulator like Duckietown, one set of parameters could make the sky purple instead of blue, or the lane markings have slightly different geometric properties, etc. The idea behind this approach is that the agent will be trained on a distribution of datasets that are all slightly different, hopefully making the agent more robust to real world nuisances once deployed in a physical body.
In this paper, the authors investigate specifically visual domain randomization.
Learn about RL, navigation, and other robot autonomy topics at the link below!
[ Duckietown Educational Resources ](https://duckietown.com/educational-resources/#machine-learning)
## Abstract
In order to train reinforcement learning algorithms, a significant amount of experience is required, so it is common practice to train them in simulation, even when they are intended to be applied in the real world. To improve robustness, camerabased agents can be trained using visual domain randomization, which involves changing the visual characteristics of the simulator between training episodes in order to improve their resilience to visual changes in their environment.
In this work, we propose a method, which includes realworld images alongside visual domain randomization in the reinforcement learning training procedure to further enhance the performance after sim-to-real transfer. We train variational autoencoders using both real and simulated frames, and the representations produced by the encoders are then used to train reinforcement learning agents.
The proposed method is evaluated against a variety of baselines, including direct and indirect visual domain randomization, end-to-end reinforcement learning, and supervised and unsupervised state representation learning.
By controlling a differential drive vehicle using only camera images, the method is tested in the Duckietown self-driving car environment. We demonstrate through our experimental results that our method improves learnt representation effectiveness and robustness by achieving the best performance of all tested methods.
## Highlights - Enhancing Visual Domain Randomization with Real Images for Sim-to-Real Transfer
Here is a visual tour of the work of the authors. For more details, check out the [full paper](#links "Post resources").
![Example frames demonstrating visual domain randomization in the Duckietown Gym self-driving simulation environment, featuring varied visual characteristics to enhance training robustness.]()
Figure 1. Example frames with visual domain randomization from the Duckietown Gym self-driving simulated environment.
![Diagram illustrating the proposed method that combines real images with visually randomized simulated images in unsupervised state representation learning, followed by training a control agent with reinforcement learning using a pretrained encoder network.]()
Figure 2. High level overview of the proposed method.
![A block diagram showing the performance of direct versus invariance regularized domain randomization and supervised versus self-supervised state representation learning methods.]()
Figure 3. Benchmarked Baseline Methods for Domain Randomization and State Representation Learning.
![Diagram showing the preprocessing pipeline.]()
Figure 4. An illustration of the preprocessing pipeline.
![Images from the offline dataset displaying three distinct renderings of the same scene, showcasing varied visual perspectives.]()
Figure 5. Samples from the Offline Dataset with Three Different Renderings for Each Scene.
![Table of evaluation results in the simulator without visual domain randomization, highlighting the proposed method in bold at the bottom row and underlining completion ratios above 70%.]()
Table 1. Evaluation Results in the Simulator Without Visual Domain Randomization.
![Table showing evaluation results in the simulator with visual domain randomization, underlining completion ratios above 70% and highlighting the proposed method in bold at the bottom row.]()
Table 2. Evaluation Results in the Simulator with Visual Domain Randomization.
![Table displaying evaluation results from real-world testing, underlining survival times of 20 or more and featuring the proposed method in bold at the bottom row.]()
Table 3. Evaluation Results in Real-World Testing.
## Conclusion - Enhancing Visual Domain Randomization with Real Images for Sim-to-Real Transfer
Here are the conclusions from the authors of this paper:
“In this work we proposed a novel method for learning effective image representations for reinforcement learning, whose core idea is to train a variational autoencoder using visually randomized images from the simulator, but include images from the real world as well, as if it was just another visually different version of the simulator.
We evaluated the method in the Duckietown self-driving environment on the lane-following task, and our experimental results showed that the image representations of our proposed method improved the performance of the tested reinforcement learning agents both in simulation and reality. This demonstrates the effectiveness and robustness of the representations learned by the proposed method. We benchmarked our method against a wide range of baselines, and the proposed method performed among the best in all cases.
Our experiments showed that using some type of visual domain randomization is necessary for a successful simto- real transfer. Variational autoencoder-based representations tended to outperform supervised representations, and both outperformed representations learned during end-to-end reinforcement learning. Also, for visual domain randomization, when using no real images, invariance regularization-based methods seemed to outperform direct methods. Based on our results, we conclude that including real images in simulation-based reinforcement learning trainings is able to enhance the real world performance of the agent – when using the two-stage approach, proposed in this paper.”
#### Project Authors
[  ](https://www.linkedin.com/in/andras-beres-789190210/)
[András Béres](https://www.linkedin.com/in/andras-beres-789190210/) is currently working as a Junior Deep Learning Engineer at [Continental](https://www.continental.com/en/), Hungary.
[  ](https://www.linkedin.com/in/gyires-toth/?originalSubdomain=hu)
Bálint Gyires-Tóth is an associate professor at
[Budapest University of Technology and Economics](http://www.bme.hu/), Hungary.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Learn about Duckietown ](http://www.duckietown.com)
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** AI, research
---
### [Reward Consistency for Interpretable Feature Discovery in RL](https://duckietown.com/leveraging-reward-consistency-for-interpretable-feature-discovery-in-reinforcement-learning/)
**Published:** July 13, 2024
**Author:** Duckietown Admin
**Excerpt:** "RL-in-RL" is proposed for interpretable feature discovery in RL agents, emphasizing reward consistency over action matching. Validated on Atari and Duckietown.
**Content:**
##### General Information
- Leveraging Reward Consistency for Interpretable Feature Discovery in Reinforcement Learning
- Qisen Yang, Huanqian Wang, Mukun Tong, Wenjie Shi, Gao Huang, and Shiji Song
- Tsinghua University, China
- Q. Yang, H. Wang, M. Tong, W. Shi, G. Huang and S. Song, "Leveraging Reward Consistency for Interpretable Feature Discovery in Reinforcement Learning," in IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 54, no. 2, pp. 1014-1025, Feb. 2024, doi: 10.1109/TSMC.2023.3312411.
[ Paper ](https://ieeexplore.ieee.org/abstract/document/10306329)
[ University ](https://www.tsinghua.edu.cn/en/)
[ Authors ](#authors)
# Leveraging Reward Consistency for Interpretable Feature Discovery in Reinforcement Learning
What is interpretable feature discovery in reinforcement learning?
To understand this, let’s introduce a few important topics:
**Reinforcement Learning (RL):** A machine learning approach where an agent gains the ability to make decisions by engaging with an environment to accomplish a specific objective. Interpretable Feature Discovery in RL is an approach that aims to make the decision-making process of RL agents more understandable to humans.
**The need for interpretability:** In real-world applications, especially in safety-critical domains like self-driving cars, it is crucial to understand why an RL agent makes a certain decision. Interpretability helps:
- Build trust in the system
- Debug and improve the model
- Ensure compliance with regulations and ethical standards
- Understand fault if accidents arise
**Feature discovery:** Feature discovery in this context refers to identifying the key artifacts (features) of the environment that the RL agent is focusing on while making decisions. For example, in a self-driving car simulation, relevant features might include the position of other cars, road signs, or lane markings.
Learn about RL, navigation, and other robot autonomy topics at the link below!
[ Duckietown Educational Resources ](https://duckietown.com/educational-resources/#machine-learning)
## Abstract
The black-box nature of deep reinforcement learning (RL) hinders them from real-world applications. Therefore, interpreting and explaining RL agents have been active research topics in recent years. Existing methods for post-hoc explanations usually adopt the action matching principle to enable an easy understanding of vision-based RL agents. In this article, it is argued that the commonly used action matching principle is more like an explanation of deep neural networks (DNNs) than the interpretation of RL agents.
It may lead to irrelevant or misplaced feature attribution when different DNNs’ outputs lead to the same rewards or different rewards result from the same outputs. Therefore, we propose to consider rewards, the essential objective of RL agents, as the essential objective of interpreting RL agents as well. To ensure reward consistency during interpretable feature discovery, a novel framework (RL interpreting RL, denoted as RL-in-RL) is proposed to solve the gradient disconnection from actions to rewards.
We verify and evaluate our method on the Atari 2600 games as well as Duckietown, a challenging self-driving car simulator environment. The results show that our method manages to keep reward (or return) consistency and achieves high-quality feature attribution. Further, a series of analytical experiments validate our assumption of the action matching principle’s limitations.
## Highlights - Leveraging Reward Consistency for Interpretable Feature Discovery in Reinforcement Learning
Here is a visual tour of the work of the authors. For more details, check out the [full paper](#links "Post resources").
![Figure showing the limitations of action matching: In the driving task illustrated in (a), two actions—turning left by 0.12 as shown in (a.1) and turning left by 0.15 as shown in (a.2)—represent the same behavior of "avoiding collision" and receive the same reward. In the scenario depicted in (b), the same actions—shown in (b.1) and (b.2)—result in different rewards depending on the task, but action matching methods would provide the same explanation for both.]()
Figure 1. Examples of action matching’s limitations.
![Diagram of the reward-oriented interpretation method: a DNN with an encoder and decoder learns the mask and attentive state. The pretrained policy takes actions from both primitive and attentive states, with the environment providing corresponding rewards. Reward matching is hindered by disconnected gradient backward in supervised learning.]()
Figure 2. Architecture of Reward-Oriented Interpretation Method for Reinforcement Learning.
![Diagram of interaction process: (a) shows how the pretrained policy (πpre) interacts with the environment during pretraining. (b) depicts the interpretation task where reward matching is treated as an RL problem, with the RL-in-RL policy (π̃) corresponding to the gray box in Fig. 2. The reward (r̃) is provided by both the environment (E) and the pretrained policy (πpre).]()
Figure 3. Interaction Process in Pretraining and Interpretation Tasks.
![𝖠𝗅𝗀𝗈𝗋𝗂𝗍𝗁𝗆 𝟣 𝗈𝗎𝗍𝗅𝗂𝗇𝖾𝗌 𝗍𝗁𝖾 𝖱𝖫-𝗂𝗇-𝖱𝖫 𝗆𝗈𝖽𝖾𝗅, 𝗐𝗁𝖾𝗋𝖾 𝖺 𝗉𝗋𝖾𝗍𝗋𝖺𝗂𝗇𝖾𝖽 𝗉𝗈𝗅𝗂𝖼𝗒 𝜋 𝗉𝗋𝖾 π 𝗉𝗋𝖾 𝗂𝗇𝗍𝖾𝗋𝖺𝖼𝗍𝗌 𝗐𝗂𝗍𝗁 𝗍𝗁𝖾 𝖾𝗇𝗏𝗂𝗋𝗈𝗇𝗆𝖾𝗇𝗍 𝗍𝗈 𝖻𝖾 𝗂𝗇𝗍𝖾𝗋𝗉𝗋𝖾𝗍𝖾𝖽. 𝖳𝗁𝖾 𝗋𝖾𝗐𝖺𝗋𝖽 𝖿𝗎𝗇𝖼𝗍𝗂𝗈𝗇 𝑅 ( 𝑠 𝑡 , 𝑎 𝑡 ) 𝖱(𝗌 𝗍 ,𝖺 𝗍 ) 𝗂𝗌 𝗉𝗋𝗈𝗏𝗂𝖽𝖾𝖽 𝖻𝗒 𝗍𝗁𝖾 𝖾𝗇𝗏𝗂𝗋𝗈𝗇𝗆𝖾𝗇𝗍 𝐸 𝖤. 𝖳𝗁𝖾 𝖺𝗅𝗀𝗈𝗋𝗂𝗍𝗁𝗆 𝗂𝗇𝗏𝗈𝗅𝗏𝖾𝗌 𝗂𝗇𝗂𝗍𝗂𝖺𝗅𝗂𝗓𝗂𝗇𝗀 𝖺𝗇𝖽 𝗎𝗉𝖽𝖺𝗍𝗂𝗇𝗀 𝗌𝗉𝖾𝖼𝗂𝖿𝗂𝖼 𝖿𝗎𝗇𝖼𝗍𝗂𝗈𝗇𝗌 𝖺𝗇𝖽 𝗉𝖺𝗋𝖺𝗆𝖾𝗍𝖾𝗋𝗌, 𝗂𝗍𝖾𝗋𝖺𝗍𝗂𝗇𝗀 𝗈𝗏𝖾𝗋 𝖾𝗉𝗈𝖼𝗁𝗌 𝗎𝗇𝗍𝗂𝗅 𝖼𝗈𝗇𝗏𝖾𝗋𝗀𝖾𝗇𝖼𝖾. 𝖣𝗎𝗋𝗂𝗇𝗀 𝖾𝖺𝖼𝗁 𝖾𝗉𝗈𝖼𝗁, 𝗍𝗁𝖾 𝗌𝗍𝖺𝗍𝖾 𝗂𝗌 𝗂𝗇𝗂𝗍𝗂𝖺𝗅𝗂𝗓𝖾𝖽, 𝖺𝖼𝗍𝗂𝗈𝗇𝗌 𝖺𝗇𝖽 𝗋𝖾𝗐𝖺𝗋𝖽𝗌 𝖺𝗋𝖾 𝖼𝗈𝗆𝗉𝗎𝗍𝖾𝖽, 𝖺𝗇𝖽 𝗍𝗋𝖺𝗃𝖾𝖼𝗍𝗈𝗋𝗂𝖾𝗌 𝖺𝗋𝖾 𝗌𝖺𝗏𝖾𝖽. 𝖳𝗁𝖾 𝗎𝗉𝖽𝖺𝗍𝖾 𝗌𝗍𝖾𝗉 𝖾𝗆𝗉𝗅𝗈𝗒𝗌 𝗍𝗁𝖾 𝖯𝗋𝗈𝗑𝗂𝗆𝖺𝗅 𝖯𝗈𝗅𝗂𝖼𝗒 𝖮𝗉𝗍𝗂𝗆𝗂𝗓𝖺𝗍𝗂𝗈𝗇 (𝖯𝖯𝖮) 𝖺𝗅𝗀𝗈𝗋𝗂𝗍𝗁𝗆 𝗍𝗈 𝗆𝖺𝗑𝗂𝗆𝗂𝗓𝖾 𝖺 𝗌𝗉𝖾𝖼𝗂𝖿𝗂𝖾𝖽 𝗈𝖻𝗃𝖾𝖼𝗍𝗂𝗏𝖾.]()
Algorithm 1. RL-in-RL Model.
![The attentive state is an overlaid combination of the state and the attentive heatmap of feature attribution. The feature importance ranges from 0 to 1 as the heatmap color changes from blue to red.]()
Figure 4. Performance of the proposed RL-in-RL model in the Duckietown environment.
![Attentive features of RL-in-RL^K with observation length K = 10]()
Figure 5. Attentive features of RL-in-RL^K with observation length K = 10.
![Comparison of interpretation methods on Atari 2600 games. The RL-in-RL model and the supervision-based method are shown with overlaid heatmaps. Perturbation-based and gradient-based methods highlight attention areas on the saliency-overlaid state. Subfigures include (a) Pong, (b) Ms. Pac-Man, and (c) Space Invaders.]()
Figure 6. Comparisons among different interpretation methods on Atari 2600.
![Comparisons between the best-performing action matching method and our reward-oriented RL-in-RL on the Duckietown environment.]()
Figure 7. Comparisons between the best-performing action matching method and our reward-oriented RL-in-RL on the Duckietown environment.
![(a) shows the percent episode return of the pretrained policy under various lane patterns, illustrating the impact of different lines (Fleft_w, Fy, Fright_w) on policy performance. (b) depicts the percent action divergence of the interpretation model under these lane patterns, comparing action consistency with the pretrained policy's actions and showing the effect of the lines on action consistency.]()
Figure 8. Quantitative analyses averaged across 100 random seeds.
![Interpretation results from the action matching method, when the policy is pretrained without the middle yellow line.]()
Figure 9. Interpretation results from the action matching method, when the policy is pretrained without the middle yellow line.
![Attention pattern of RL−in−RLᵃ.]()
Figure 10. Attention pattern of RL−in−RLᵃ.
![Architecture of the encoder–decoder network in RL-in-RL]()
Figure 11. Architecture of the encoder–decoder network in RL-in-RL.
![Additional results of the RL-in-RL model on Atari 2600 games. Attentive states are visualized as in Figure 4. The games displayed are: (a) Enduro, (b) Assault, and (c) Breakout.]()
Figure 12. More results of the RL-in-RL model on Atari2600.
![𝖳𝗐𝗈 𝖺𝖻𝗅𝖺𝗍𝗂𝗈𝗇 𝗌𝗍𝗎𝖽𝗂𝖾𝗌 𝗈𝗇 𝗍𝗁𝖾 𝗁𝗒𝗉𝖾𝗋𝗉𝖺𝗋𝖺𝗆𝖾𝗍𝖾𝗋𝗌 𝗈𝖿 𝗍𝗁𝖾 𝖱𝖫-𝗂𝗇-𝖱𝖫 𝗆𝗈𝖽𝖾𝗅. (𝖺) 𝖲𝗁𝗈𝗐𝗌 𝗍𝗁𝖾 𝖾𝖿𝖿𝖾𝖼𝗍 𝗈𝖿 𝗌𝖾𝗍𝗍𝗂𝗇𝗀 𝛼 = 𝟢.𝟣 α=𝟢.𝟣, 𝗐𝗁𝖾𝗋𝖾 𝛼 α 𝗂𝗌 𝗍𝗁𝖾 𝗐𝖾𝗂𝗀𝗁𝗍 𝗈𝖿 𝗍𝗁𝖾 𝖺𝗎𝗑𝗂𝗅𝗂𝖺𝗋𝗒 𝗍𝖺𝗌𝗄 𝗅𝗈𝗌𝗌 𝖺𝗌 𝖽𝖾𝖿𝗂𝗇𝖾𝖽 𝗂𝗇 𝖤𝗊𝗎𝖺𝗍𝗂𝗈𝗇 (𝟪). (𝖻) 𝖲𝗁𝗈𝗐𝗌 𝗍𝗁𝖾 𝖾𝖿𝖿𝖾𝖼𝗍 𝗈𝖿 𝗌𝖾𝗍𝗍𝗂𝗇𝗀 𝛽 = 𝟢.𝟣 β=𝟢.𝟣, 𝗐𝗁𝖾𝗋𝖾 𝛽 β 𝗂𝗌 𝗎𝗌𝖾𝖽 𝗍𝗈 𝗌𝗉𝖾𝖾𝖽 𝗎𝗉 𝗍𝗁𝖾 𝗍𝗋𝖺𝗂𝗇𝗂𝗇𝗀 𝗉𝗋𝗈𝖼𝖾𝗌𝗌 𝖺𝗌 𝖽𝖾𝗌𝖼𝗋𝗂𝖻𝖾𝖽 𝗂𝗇 𝖠𝗉𝗉𝖾𝗇𝖽𝗂𝗑 𝖠.]()
Figure 13. Two ablation studies on the hyperparameters.
![Comparative results of the RL-in-RL method and the action matching method’s ablation studies on the hyperparameter α.]()
Figure 14. Comparative results of the RL-in-RL method and the action matching method’s ablation studies on the hyperparameter α.
![Visulization of attention shift in the zigzag turn. Three rows correspond to original states, attentive states, and masked states, respectively.]()
Figure 15. Visulization of attention shift in the zigzag turn.
## Conclusion
Here are the conclusions from the authors of this paper:
“In this article, we discussed the limitations of the commonly used assumption, the action matching principle, in RL interpretation methods. It is suggested that action matching cannot truly interpret the agent since it differs from the reward-oriented goal of RL. Hence, the proposed method first leverages reward consistency during feature attribution and models the interpretation problem as a new RL problem, denoted as RL-in-RL.
Moreover, it provides an adjustable observation length for one-step reward or multistep reward (or return) consistency, depending on the requirements of behavior analyses. Extensive experiments validate the proposed model and support our concerns that action matching would lead to redundant and noncausal attention during interpretation since it is dedicated to exactly identical actions and thus results in a sort of “overfitting.”
Nevertheless, although RL-in-RL shows superior interpretability and dispenses with redundant attention, further exploration of interpreting RL tasks with explicit causality is left for future work.”
#### Project Authors
[  ](https://qisen-yang.netlify.app/)
[Qisen Yang](https://qisen-yang.netlify.app/) is an Artificial Intelligence PhD Student at [Tsinghua University](https://www.tsinghua.edu.cn/en/), China.
[  ](https://ieeexplore.ieee.org/author/920398723103973)
[Huanqian Wang](https://ieeexplore.ieee.org/author/920398723103973) is currently pursuing the B.E. degree in control science and engineering with the Department of Automation, [Tsinghua University](https://www.tsinghua.edu.cn/en/), Beijing, China.
[  ](https://ieeexplore.ieee.org/author/37089895948)
[Mukun Tong](https://ieeexplore.ieee.org/author/37089895948) is currently pursuing the B.E. degree in control science and engineering with the Department of Automation, [Tsinghua University](https://www.tsinghua.edu.cn/en/),
Beijing, China.
[  ](https://ieeexplore.ieee.org/author/37086608992)
[Wenjie Shi](https://ieeexplore.ieee.org/author/37086608992) received his Ph.D. degree in control science and engineering from the Department of Automation, Institute of Industrial Intelligence and System, [Tsinghua University](https://www.tsinghua.edu.cn/en/), Beijing, China, in 2022.
[  ](https://ieeexplore.ieee.org/author/37274328900)
[Guang-Bin Huang](https://ieeexplore.ieee.org/author/37274328900) is in the School of Electrical and Electronic Engineering, [Nanyang Technological University](https://www.ntu.edu.sg/), Singapore.
[  ](https://ieeexplore.ieee.org/author/37332904400)
[Shiji Song](https://ieeexplore.ieee.org/author/37332904400) is currently a Professor with the Department of Automation, [Tsinghua University](https://www.tsinghua.edu.cn/en/), Beijing, China.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Learn about Duckietown ](http://www.duckietown.com)
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** AI, research
---
### [Vision-Based DRL Autonomous Driving Agent with Sim2Real Transfer](https://duckietown.com/vision-based-drl-autonomous-driving-agent-with-sim2real-transfer-li-okhrin/)
**Published:** May 6, 2024
**Author:** Duckietown Admin
**Excerpt:** To achieve autonomous driving, vehicles must perform tasks such as lane and car following. Here is a vision-based deep RL agent with Sim2Real training to do so.
**Content:**
##### General Information
- Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer
- Dianzhao Li and Ostap Okhrin
- Universität Dresden, Dresden
- D. Li and O. Okhrin, "Vision-Based DRL Autonomous Driving Agent with Sim2Real Transfer," 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), Bilbao, Spain, 2023, pp. 866-873, doi: 10.1109/ITSC57777.2023.10422677.
[ Paper (ITSC) ](https://ieeexplore.ieee.org/abstract/document/10422677)
[ Paper (ArXiv) ](https://arxiv.org/abs/2305.11589)
[ Code ](https://github.com/DailyL/Sim2Real_autonomous_vehicle)
[ University ](https://tu-dresden.de/)
[ Authors ](#authors)
# Vision-Based DRL Autonomous Driving Agent with Sim2Real Transfer
One way to obtain quick and cheap training data is to use simulation instead of real-world experiments. The question remains if the learnings of a simulation-trained agent apply to the real world. Sim2Real transfer is the field of research that studies this problem.
The challenge is particularly meaningful when using vision as the primary sensing capability for robots. Vision-based deep reinforcement learning (DRL) refers to a technique where ML agents, typically modeled as multi-layered neural networks, learn to “make decisions” directly from visual input.
The essence of RL is training robotic agents based on policies that reward desirable outcomes. This family of techniques typically leads to increased adaptability to operational scenarios.
To learn about RL and its place in the larger context of robot autonomy, check out the resources below.
[ Duckietown Educational Resources ](https://duckietown.com/educational-resources/#machine-learning)
## Abstract
To achieve fully autonomous driving, vehicles must be capable of continuously performing various driving tasks, including lane keeping and car following, both of which are fundamental and well-studied driving ones. However, previous studies have mainly focused on individual tasks, and car following tasks have typically relied on complete leader-follower information to attain optimal performance.
To address this limitation, we propose a vision-based deep reinforcement learning (DRL) agent that can simultaneously perform lane-keeping and car-following maneuvers.
To evaluate the performance of our DRL agent, we compare it with a baseline controller and use various performance metrics for quantitative analysis. Furthermore, we conduct a real-world evaluation to demonstrate the Sim2Real transfer capability of the trained DRL agent.
To the best of our knowledge, our vision-based car following and lane-keeping agent with Sim2Real transfer capability is the first of its kind.
## Highlights - Sim2Real transfer results
Here is a visual tour of the work of the authors. For all the details, check out the [paper link](#links).
Fig. 1. The proposed DRL framework for vision-based multi-task autonomous driving agents. The perception module leverages camera images to produce impact attributes regarding the environment, then the DRL control module utilizes the information to control the agent with enhanced generalization.
Fig. 2. Robot car \[Duckiebot\] used during the real-world evaluation. (a) Side view of the robot car which equipped with front-view camera and Jetson Nano 2GB. (b) Back view of the car with a pattern of circles.
Fig. 3. An example velocity trajectory for the leading vehicle generated by the Ornstein-Uhlenbeck process for the training process.
Fig. 4. Training results of PPO agent with ten independent seeds over one million steps.
Fig. 5. Example trajectories of DRL agent (red) and baseline controller (green) following random leader trajectory (orange) during the evaluation.
Fig. 6. Distribution of TTC and time headway for DRL agent (red) and baseline controller (green).
Fig. 7. Example trajectories of DRL agent (red) and baseline controller (green) following a self defined external leader trajectory (orange) during the evaluation.
Fig. 8. Distribution of TTC and time headway for DRL agent (red) and baseline controller (green).
## Conclusion
This study proposes a vision-based DRL agent that can simultaneously perform lane-keeping and car-following tasks.
The overall system is divided into two modules: the perception module and the control module. The perception module extracts task-relevant attributes of the surroundings, while the control module is a DRL agent that takes these attributes as input. To evaluate the performance of the DRL agent, we compare it with a baseline algorithm in both simulation and real-world environments.
In the simulation, we compare the car following and lane-keeping capabilities of the DRL agent and baseline controller using various performance metrics. In the real-world environment, we demonstrate that the DRL agent can follow the leading vehicle while maintaining lane-keeping ability.
In future work, we plan to enhance our DRL agent by incorporating a comfort factor to address unstable driving behavior. Additionally, we aim to deploy more advanced algorithms for improved generalization.
## Project Authors
[  ](https://www.linkedin.com/in/suri-rohit/)
[Dianzhao Li](https://ieeexplore.ieee.org/author/388655525177583) is a research assistant at the [Technische Universität Dresden](https://tu-dresden.de/), Dresden, Germany.
[  ](https://www.linkedin.com/in/suri-rohit/)
[Ostap Okhrin](https://ieeexplore.ieee.org/author/484634543471489) is Chair of Statistics and Econometrics at the Institute of Economics and Transport, School of Transportation, [Technische Universitat Dresden](https://tu-dresden.de/) in Germany.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Learn about Duckietown ](http://www.duckietown.com)
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** AI, engineering education, research
---
### [Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm](https://duckietown.com/sequential-multi-policy-reinforcement-learning-dukkipati-et-al/)
**Published:** April 4, 2024
**Author:** Duckietown Admin
**Excerpt:** Solving complex problems using reinforcement learning necessitates breaking down the problem into manageable tasks, and learning policies to solve these tasks. Learn more about Sequential Multi-Policy Reinforcement Learning in this article.
**Content:**
##### General Information
- Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm
- Ambedkar Dukkipati, Rajarshi Banerjee, Ranga Shaarad Ayyagari, Dhaval Parmar Udaybhai
- Indian Institute of Science
- A. Dukkipati, R. Banerjee, R. S. Ayyagari and D. P. Udaybhai, "Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm," 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 2022, pp. 2483-2489, doi: 10.1109/IROS47612.2022.9981607.
[ Paper (IROS) ](https://ieeexplore.ieee.org/abstract/document/9981607)
[ Paper (ArXiv) ](https://arxiv.org/abs/2102.00168)
[ Author ](https://www.csa.iisc.ac.in/~ambedkar/)
[ University ](https://iisc.ac.in/)
# Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm
Reinforcement learning (RL) is a rising star approach for developing autonomous robot agents. The essence of RL is training agents based on policies that reward desirable outcomes, which leads to increased adaptability to operational scenarios. Through iterations, robots refine their decision-making, optimizing actions based on rewards and penalties. This method provides robots with the flexibility to handle unpredictable situations, enhancing their efficiency and effectiveness in real-world tasks. To learn about RL with Duckietown, check out the resources below.
[ Duckietown Educational Resources ](https://duckietown.com/educational-resources/#machine-learning)
## Abstract
Solving complex problems using reinforcement learning necessitates breaking down the problem into manageable tasks, and learning policies to solve these tasks. These policies, in turn, have to be controlled by a master policy that takes high-level decisions. Hence learning policies involves hierarchical decision structures. However, training such methods in practice may lead to poor generalization, with either sub-policies executing actions for too few time steps or devolving into a single policy altogether. In our work, we introduce an alternative approach to learn such skills sequentially without using an overarching hierarchical policy. We propose this method in the context of environments where a major component of the objective of a learning agent is to prolong the episode for as long as possible. We refer to our proposed method as Sequential Soft Option Critic. We demonstrate the utility of our approach on navigation and goal-based tasks in a flexible simulated 3D navigation environment that we have developed. We also show that our method outperforms prior methods such as Soft Actor-Critic and Soft Option Critic on various environments, including the Atari River Raid environment and the Gym-Duckietown self-driving car simulator.
## Highlights
Here is a visual tour of the work of the authors.
For all the details, check out the [paper link](#links)!
![(a) An illustration of policies learned in our approach. (b) A representation of the state space partitioned by the termination functions of the options. Each oval corresponds to the set of states classified as non-termination states by the corresponding nested termination function.]()
Fig. 1. (a) An illustration of policies learned in our approach. Different policies learn different skills out of necessity to traverse the environment and not terminate the episode. The trajectory spawned by the sequence of policies that correctly learns to avoid terminating the episode is shown in green. The policy π1 is used when the agent is in the middle of the corridor. When the end of the corridor is reached, π2 is selected to take a turn and avoid a collision. (b) A representation of the state space partitioned by the termination functions of the options. Each oval corresponds to the set of states classified as non-termination states by the corresponding nested termination function.
![Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm image]()
Fig. 2. (a) & (b) The input to the neural network in our 3D navigation environment. These images obtained from the simulation are scaled (and also transformed to grayscale in the case of (a)) and appended with K previous images, and sent to the agent as input. (c) A view of the Duckietown environment.
![Autonomous robot Navigation performance results obtained on various environments: SAC, SOC, SAC with HER]()
Fig. 3. The results obtained on various environments.
![Outputs of the three learned policies when fed the corresponding input shown on the left. πω1 : orange, πω2 : blue, πω3 : green. Each policy outputs a Gaussian distribution, the active policy is the one filled with color. A positive value for the output action corresponds to turning right, and a negative value indicates a left turn.]()
Fig. 4. Outputs of the three learned policies when fed the corresponding input shown on the left. πω\_1: orange, πω\_2 : blue, πω\_3 : green. Each policy outputs a Gaussian distribution, the active policy is the one filled with color. A positive value for the output action corresponds to turning right, and a negative value indicates a left turn.
## Conclusion
In this paper, the authors proposed an algorithm called “Sequential Soft Option Critic” that allows adding new skills dynamically without the need for a higher-level master policy. This can be applicable to environments where a primary component of the objective is to prolong the episode. We show that this algorithm can be used to effectively incorporate diverse skills into an overall skill set, and it outperforms prior methods in several environments.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
Duckietown is modular, customizable and state-of-the-art. It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Learn about Duckietown ](http://www.duckietown.com)
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** education, News, paper, Research
**Tags:** AI, engineering education, research
---
### [Sim2Real Transfer of Multi-Agent Policies for Self-Driving](https://duckietown.com/sim2real-transfer-of-multi-agent-policies-for-self-driving/)
**Published:** August 10, 2024
**Author:** Duckietown Admin
**Excerpt:** This work proposes a sim2real transfer method for AVs multi-agent policies, reducing the reality gap using "MAPPO" with domain randomization.
**Content:**
##### General Information
- Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Real
- Eduardo Candela, Leandro Parada, Luis Marques, Tiberiu-Andrei Georgescu, Yiannis Demiris, Panagiotis Angeloudis
- Imperial College London, United Kingdom
- E. Candela, L. Parada, L. Marques, T. -A. Georgescu, Y. Demiris and P. Angeloudis, "Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Real," 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 2022, pp. 8814-8820, doi: 10.1109/IROS47612.2022.9981319.
[ Paper ](https://ieeexplore.ieee.org/document/9981319)
[ Institution ](https://www.imperial.ac.uk/)
[ Authors ](#authors)
# Sim2Real Transfer of Multi-Agent Policies for Self-Driving
In the field of autonomous driving, transferring policies from simulation to the real world (Sim-to-real transfer, or Sim2Real) is theoretically desirable, as it is much faster and more cost-effective to train agents in simulation rather than in the real world.
Given simulations are just that – representations of the real world – the question of whether the trained policies will actually perform well enough in the real world is always open. This challenge is known as “Sim-to-Real gap”.
This gap is especially pronounced in Multi-Agent Reinforcement Learning (MARL), where agent collaboration and environmental synchronization significantly complicate policy transfer.
The authors of this work propose employing “Multi-Agent Proximal Policy Optimization” (MAPPO) in conjunction with domain randomization techniques, to create a robust pipeline for training MARL policies that is not only effective in simulation but also adaptable to real-world conditions.
Through varying levels of parameter randomization—such as altering lighting conditions, lane markings, and agent behaviors— the authors enhance the robustness of trained policies, ensuring they generalize effectively across a wide range of real-world scenarios.
Learn about training, sim2real, navigation, and other robot autonomy topics with Duckietown starting from the link below!
[ Duckietown Educational Resources ](https://duckietown.com/educational-resources/#machine-learning)
## Abstract
Autonomous Driving requires high levels of coordination and collaboration between agents. Achieving effective coordination in multi-agent systems is a difficult task that remains largely unresolved. Multi-Agent Reinforcement Learning has arisen as a powerful method to accomplish this task because it considers the interaction between agents and also allows for decentralized training—which makes it highly scalable.
However, transferring policies from simulation to the real world is a big challenge, even for single-agent applications. Multi-agent systems add additional complexities to the Sim-to-Real gap due to agent collaboration and environment synchronization.
In this paper, we propose a method to transfer multi-agent autonomous driving policies to the real world. For this, we create a multi-agent environment that imitates the dynamics of the Duckietown multi-robot testbed, and train multi-agent policies using the MAPPO algorithm with different levels of domain randomization. We then transfer the trained policies to the Duckietown testbed and show that when using our method, domain randomization can reduce the reality gap by 90%.
Moreover, we show that different levels of parameter randomization have a substantial impact on the Sim-to-Real gap. Finally, our approach achieves significantly better results than a rule-based benchmark.
## Highlights - Sim2Real Transfer of Multi-Agent Policies for Self-Driving
Here is a visual tour of the work of the authors. For more details, check out the [full paper](#links "Post resources").
![Image of a test environment where autonomous Duckiebot robots navigate a track while avoiding parked obstacles. The setup includes a motion capture camera system tracking the real-time pose of the vehicles.]()
Figure 1. Autonomous Vehicle Training on a Real-World Track Using Multi-Agent Deep Reinforcement Learning with Sim2Real Transfer.
![Diagram of a test track used in both simulation and real-world settings, showing waypoints, robot positions, goal points, and the steering angle required for path following in the Duckie-MAAD environment.]()
Figure 2. Test Track for Duckie-MAAD Gym Environment with Waypoints and Path Following Function.
![Flowchart of the Duckie-MAAD architecture showing the step update loop for multi-agent autonomous driving, including action selection, path following, inverse kinematics, domain randomization, and policy updates with MAPPO.]()
Figure 3. Duckie-MAAD Architecture: Step Update Loop for Sim2Real Transfer in Multi-Agent Autonomous Driving.
![Graph showing the reward convergence over time during the training of a Medium Domain Randomization (D.R.) policy in a multi-agent reinforcement learning setup.]()
Figure 4. Reward Convergence During Training of Medium Domain Randomization (D.R.) Policy.
![Bar chart comparing average rewards for multi-agent reinforcement learning (MARL) policies in simulation and real life, showing the effectiveness of medium domain randomization in bridging the Sim2Real gap.]()
Figure 5. Comparison of Average Rewards for MARL Policies in Simulation and Real Life.
![Box plots comparing the distribution of speed and track exit metrics across different policies, including rule-based and MARL policies with varying levels of domain randomization (D.R.), in both simulation and real-world scenarios.]()
Figure 6. Performance Metrics Distribution for Various Policies: Speed and Track Exits.
![Figure 7. Performance Metrics Distribution: Collisions and Lane Changes Across Different Policies.]()
Figure 7. Performance Metrics Distribution: Collisions and Lane Changes Across Different Policies.
## Conclusion - Sim2Real Transfer of Multi-Agent Policies for Self-Driving
Here are the conclusions from the authors of this paper:
“AVs will lead to enormous safety and efficiency benefits across multiple fields, once the complex problem of multiagent coordination and collaboration is solved. MARL can help towards this, as it enables agents to learn to collaborate by sharing observations and rewards.
However, the successful application of MARL, is heavily dependent on the fidelity of the simulation environment they were trained in. We present a method to train policies using MARL and to reduce the reality gap when transferring them to the real world via adding domain randomization during training, which we show has a significant and positive impact in real performance compared to rule-based methods or policies trained without different levels of domain randomization.
It is important to mention that despite the performance improvements observed when using domain randomization, its use presents diminishing returns as seen with the overly conservative policy, for it cannot completely close the reality gap without increasing the fidelity of the simulator. Additionally, the amount of domain randomization to be used is case-specific and a theory for the selection of domain randomization remains an open question. The quantification and description of reality gaps presents another opportunity for future research.”
#### Project Authors
[Eduardo Candela](https://www.linkedin.com/in/eduardo-candela-garza/) is currently working as the Co-Founder & CTO of [MAIHEM (YC W24)](https://www.maihem.ai/), California.
[Leandro Parada](https://www.linkedin.com/in/leandro-parada-p/) is a Research Associate at [Imperial College London](https://www.imperial.ac.uk/), United Kingdom.
[  ](https://www.linkedin.com/in/gyires-toth/?originalSubdomain=hu)
[Luís Marques](https://www.linkedin.com/in/luis-marques-lfsm/) is a Doctoral Researcher in the Department of Robotics at the [University of Michigan](https://umich.edu/), USA.
[Tiberiu Andrei Georgescu](https://www.linkedin.com/in/tibigeo/) is a Doctoral Researcher at [Imperial College London](https://www.imperial.ac.uk/), United Kingdom.
[Yiannis Demiris ](https://www.linkedin.com/in/yiannisdemiris/)is a Professor of Human-Centred Robotics and Royal Academy of Engineering Chair in Emerging Technologies at [Imperial College London](https://www.imperial.ac.uk/), United Kingdom.
[Panagiotis Angeloudis](https://www.linkedin.com/in/panagiotisangeloudis/) is a Reader in Transport Systems and Logistics at [Imperial College London](https://www.imperial.ac.uk/), United Kingdom.
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Learn about Duckietown ](http://www.duckietown.com)
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**Categories:** paper, Research
**Tags:** AI, research
---
### [Exploring the skies at the summer Duckiedrone academy](https://duckietown.com/exploring-the-skies-at-the-summer-duckiedrone-academy/)
**Published:** July 27, 2023
**Author:** Ivano Marocchi
**Excerpt:** The Duckiedrone summer academy is an event organized in Boston by MassRobotics, Brown University and Duckietown with the generous support of Amazon Robotics designed to introduce young learners to autonomous robotics.
**Content:**
**Boston, 7-11 July 2023**: Congratulations to the fourth cohort of students of the Duckiedrone summer academy, hosted by Massrobotics, Brown University and Duckietown with the generous support of Amazon Robotics!
##### Quick links
- [ Massrobotics Drone Academy ](https://www.massrobotics.org/stem/summer-drone-acadamy/)
- [ Brown University Duckiesky ](https://sites.brown.edu/duckiesky/)
- [ Duckiedrone (DD21) Operational Manual ](https://www.linkedin.com/in/dr-debasis-das-42839a26/?originalSubdomain=in)
- [ Instructor manual ](https://docs.duckietown.com/daffy/course-intro-to-drones/intro.html)
- [ Get a Duckiedrone ](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24)
- [ Reach out to teach your Duckiedrone class ](https://contact.duckietown.com/request-a-quote)
## Exploring the skies at the summer Duckiedrone academy
As the sun shines high, the summer Duckiedrone academy, a program which sees the cooperation of Duckietown, Amazon Fulfillment Technology and Robotics, MassRobotics and Brown University, has attracted high school students from the greater Boston area to dive into the world of autonomous aerial vehicles.
The **Duckiedrone** is a DIY, open, Raspberry Pi-based quadcopter kit designed for introducing learners to autonomous flight. Comes with a polished undergraduate-level course and the support of the Duckietown international community.
[ Get the Duckiedrone ](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24)
[ Build the Duckiedrone ](https://docs.duckietown.com/daffy/opmanual-duckiedrone/intro.html)
[ Make it fly ](https://docs.duckietown.com/daffy/course-intro-to-drones/intro.html)
Students learned how to build, program, and fly a drone starting from a box of components, in addition to participating in workshops held by industry professionals such as Stephanie Tellex, Associate Professor of Computer Science at Brown University, and Andrea Francesco Daniele, Chief Technology Officer at Duckietown.
In recent years autonomous robots have started revolutionizing many industries, and drones are playing an important role in this ongoing trend with applications from agriculture to inspection, surveillance, and warehouse management.
These versatile flying machines are a gateway to the fundamentals of robot autonomy, especially (but not only!) for younger learners. Seeing a machine fly on its own is exciting!
The Duckiedrone comes with step-by-step instructions for assembly, calibration, manual and autonomous operations. Students learn from the basics of mechatronics, such as soldering and handling of electrical circuits, to elements of autonomy including sensor calibration, middlewares (ROS), PID control, online filtering and simultaneous localization and mapping (SLAM) using Python and interactive Jupyter notebooks.
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
[ Request a quote for your class ](https://contact.duckietown.com/request-a-quote)
[  ](https://www.massrobotics.org/stem/summer-drone-acadamy/)
[  ](https://sites.brown.edu/duckiesky/)
**Categories:** Blog, editor-choiche, Events, News
---
### [Teaching robot autonomy at AGH Krakow with Prof. Długosz](https://duckietown.com/dlugosz-robot-autonomy/)
**Published:** June 21, 2024
**Author:** Federico Tani
**Excerpt:** Prof. Marek Dlugosz tells us what he likes about teaching robot autonomy with Duckietown in his AI and Autonomous Vehicles laboratory in Krakow, Poland.
**Content:**
# Teaching robot autonomy at AGH Krakow with Prof. Długosz
Prof. Marek Dlugosz tells us what he likes about teaching robot autonomy with Duckietown in his AI and Autonomous Vehicles laboratory in Krakow, Poland.
**Krakow, Poland, April 29th 2024**
Professor Marek Długosz from the Akademia Górniczo-Hutnicza (AGH) – University of Science and Technology in Krakow, shares his experience teaching robot autonomy with Duckietown.
##### Quick links
- [ Marek Dlugosz on Google Scholar ](https://scholar.google.pl/citations?hl=pl&user=XFIEzpsRqNIC)
- [ AGH University ](https://www.agh.edu.pl/en/)
- [ Centre of Excellence in Artifical Intelligence ](https://ceai.agh.edu.pl/)
- [ AGH Computer Vision Laboratory ](https://www.linkedin.com/company/vision-agh/?originalSubdomain=pl)
## Increasing efficiency and saving time at the robot autonomy lab in AGH Krakow
We interviewed Prof. Marek Długosz form the AGH University of Science and Technology in Krakow, to learn more about his teaching activities and laboratory.
##### Good morning and thank you for taking the time to be with us! Could you introduce yourself?
My name is Marek Długosz and I am an assistant professor at the AGH University of Science and Technology in Krakow. My work is focused on robotic automatic functions theory, while last year I focused more on robot autonomy and autonomous vehicles.
##### Thank you. When have you encountered Duckietown for the first time?
I first learned about the Duckietown project while searching the internet for information, then I started to read more about it, and it looked very interesting for a few reasons. First of all, there’s not only a theoretical part, but there’s also the hardware part of this project which are the Duckiebots.
Then, there’s also a lot of lessons, examples and exercises for students to learn more about robotics. That’s one reason why I found it interesting. A second reason this project looked interesting is that it’s not only about one robot, but a fleet of robots, and also there is the speed regulation aspect. One of the main focus of my research is how to manage the speed of autonomous robots or autonomous cars, and I think that Duckietown can be perfect to check my ideas and algorithms rapidly and with ease. It is much easier to run five robots in my laboratory than to run four or five cars in reality!
##### How do you use Duckietown in your activities?
My colleagues and I use Duckiebots during classes and lessons to better explain topics related to theoretical aspects of programming in robotics. What I find very attractive is the possibility for my students to practice. They can actually program a Duckiebot and implement algorithms, such as lane following, adaptive cruise control etc. This is really something that my students and colleagues like very much. Duckietown also has fantastic software, it’s very well organized, thanks to Docker, so there is no risk that some student makes a mistake and breaks one of my robots! Students prepare the Docker container, and once the exercise is complete, they delete this container.
We also use Duckietown in our research as I said earlier, to verify and check our algorithms, and how to manage fleets of robots. One of the special aspects of Duckietown are its smart cities, with its crossroads, signs, traffic lights, etc. For us, this aspect is very interesting.
One of the latest projects done by my students was a system to localize Duckiebots on a plane using four or more video cameras, and we are about to publish the results of this project.
##### Very interesting thank you. What is the age of the students you are teaching right now, and would you say the students are satisfied with Duckietown?
The age bracket is students between 20 and 23 years old. I’ll just say that very often, at the end of the lessons, students decide to stay and do more exercises, they find it incredibly interesting that Duckietown is not only a simulation, as we know in a simulation you can do anything, but when you start doing things practically, taking the hardware in your hands, and programming these robots, it’s a very, very different thing.
Very often I have to stay after lessons and try to do more together with my students, to the point I must tell them to come back the day after because it’s getting late.
Also, in addition to regular lessons, in our university there is a Student Scientist Association, the members of which can go to special classes after the end of regular ones, to perform additional experiments and exercises.
> Duckietown is not only a simulation, in a simulation you can do anything, but when you start doing things practically, taking the hardware in your hands, and programming these robots, it’s a very, very different thing.
>
> Marek Długosz
##### Did you encounter any challenges, problems or difficulties while using Duckietown?
At the beginning, we had some problems with the assembly of the Duckiebot. I even wrote a simple article about this, but we made some improvements, which I described in this [article](https://ieeexplore.ieee.org/abstract/document/10343846), and now the Duckiebots work perfectly.
##### Do you have anything else to add about your projects and experiences?
I hope that I can motivate enough students to participate in the AI-DO competition. I’d like it if some of my students participated in that kind of challenge. I would also like to let my students understand how well organized Duckietown software is, I think there’s an absolutely perfect architecture that helps minimize the risk of errors and mistakes and there’s perfect functionality between containers. We have a lot of students and these robots need to work for everyone, so minimizing the possibility of problems is excellent.
> I think that Duckietown can be perfect to check my ideas and algorithms rapidly and with ease. It is much easier to run five robots in my laboratory than to run four or five cars in reality!
>
> Marek Długosz
##### Would you recommend Duckietown to colleagues and students?
Yes of course, anytime I have the occasion I recommend this project. Ours is the Artificial Intelligence and Autonomous Vehicles laboratory, and it has rapidly become one of the most attractive ones in the University. Very often students show up just to see what we are doing, and every time I show Duckietown’s smart cities, and how robots drive around and stop at traffic lights, crossroads etc. It’s always a lot of fun.
### Learn more about Duckietown
Duckietown enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** News, People
**Tags:** AI-DO, duckiebot, robotics, student
---
### [Development of an Ackermann steering autonomous vehicle](https://duckietown.com/ackermann-steering-duckiebot/)
**Published:** June 12, 2024
**Author:** Duckietown Admin
**Excerpt:** This student project implements an Ackermann steering system on a Duckiebot to simulate 4 wheels, and better and more complex real-world car model.
**Content:**
# Development of an Ackermann steering autonomous vehicle
##### Project Resources
- **Objective**: The objective of this project was to create an Ackermann steering based self driving car in Duckietown.
- **Approach**: Increasing the complexity (and realism) of the hardware chassis, from differential drive to Ackermann steering.
- **Authors**: Merlin Hosner and Raphael Fröhlich
[ Final Result ](#project-result)
[ Thesis ](https://drive.google.com/file/d/1V5SUjv2Dfr2Ia3_mPUKgZF8BPMU8CNi3/view?usp=sharing)
[ Code ](https://github.com/duckietown?q=dbv2&type=all&language=&sort=)
[ Authors ](#authors)
## Why Ackermann steering?
Ackermann steering is a configuration of wheels on a vehicle charachterized by four wheels, two in the back that are powered by a DC motor, and two in the front that steer though commands received by a servo motor. In contrast, differential drive robots have two wheels that are independently powered by two DC-motors, with a passive omnidrectional third wheel that acts as support.
The dynamics (i.e., the “kind of movement”) of differential drive robots is quite different from real world automobiles, which, e.g., cannot turn on the spot. Ackerman steering achieves more realistic vehicle dynamics at cost: increased hardware complexity and mathematical modeling. But neither of these challenges have stopped talented Duckietown student from designing and implementing an Ackermann steering Duckiebot!
(Duckietown trivia: careful Duckietown observers will have noticed that the Duckiebot models historically have been called DB18, DB19, DB21, etc. – every wondered which would have been the DB20?)
## Ackermann steering in Duckietown: the challenges
Ackermann steering introduces more complex mathematical modeling, with respect to differential drive robots, in order to predict future movement hence elaborate pose estimates on the fly. The kinematic modeling of the front steering apparatus is non trivial, and the radius of curvature Ackermann steering robots showcase is very different from differential drive robots.
Differential drive robots are capable of turning on the spot (applying equal and opposite commands to the two wheels), while anyone who has ever tried parallel parking a real car, knows that this is not possible.
How complex will it be for Ackermann steering robots to navigate Duckietown is the real challenge of this fun project.
The authors start from basic design elements through CAD, iterate through various bills of materials, make prototypes, and program them leveraging the Duckietown software infrastructure to achieve autonomous behaviors in Duckietown.
## Project Highlights
Here is the output of their work. [Check out the documents](#links "project resources") for more details!
![Configuration of a differential drive]()
Configuration of a differential drive
![Configuration of a bicycle model]()
Configuration of a bicycle model
![Extension to four wheels]()
Extension to four wheels
![Trace of a four bar rigid link configuration]()
Trace of a four bar rigid link configuration
![Geometric approximation of the Ackermann criteria]()
Geometric approximation of the Ackermann criteria
![Block diagram of a back-calculation anti-windup logic]()
Block diagram of a back-calculation anti-windup logic
![DBv2 placed in a lane with the control inputs in red and the control outputs in white. The steering angle γ is calculated from the angle ω. The linear velocity is represented by v]()
DBv2 placed in a lane with the control inputs in red and the control outputs in white. The steering angle γ is calculated from the angle ω. The linear velocity is represented by v
![Our geometry for implementing the Ackermann criteria]()
Our geometry for implementing the Ackermann criteria
![Ackermann criteria met on the steering of the DBv2]()
Ackermann criteria met on the steering of the DBv2
![Ackermann criteria met on the steering of the DBv2]()
Ackermann criteria met on the steering of the DBv2
![Ackermann criteria met on the steering of the DBv2]()
Ackermann criteria met on the steering of the DBv2
## Ackermann steering: Results
(Turn on the sound for best experience!)
The autonomous behaviors of the Ackermann steering Duckiebot, a.k.a. DB20 or DBv2, shown above are the work of Timothy Scott, a former Duckietown student.
## Ackermann steering Duckiebot: Authors
[  ](https://www.linkedin.com/in/merlin-hosner-206637128/?originalSubdomain=ch)
[Merlin Hosner](https://www.linkedin.com/in/merlin-hosner-206637128/?originalSubdomain=ch) is a former Duckietown student in the Institute for Dynamic Systems and Controls (IDSC) of [ETH Zurich](https://www.linkedin.com/school/eth-zurich/) (D-MAVT), and currently works at [Climeworks](https://www.linkedin.com/company/climeworks/) as a Process Development Engineer.
[  ](https://www.linkedin.com/in/rafael-fr%C3%B6hlich-856001240/?original_referer=&originalSubdomain=ch)
[Rafael Fröhlich](https://www.linkedin.com/in/rafael-fr%C3%B6hlich-856001240/?original_referer=&originalSubdomain=ch) is a former Duckietown student in the Institute for Dynamic Systems and Controls (IDSC) of [ETH Zurich](https://www.linkedin.com/school/eth-zurich/) (D-MAVT), where he is currently a Research Assistant.
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** Projects
**Tags:** Ackermann steering, AI, autonomous vehicles, duckiebot, duckietown, engineering education, self driving car
---
### [Vision-based Reinforcement Learning for Lane-Tracking Control](https://duckietown.com/vision-based-reinforcement-learning-for-lane-tracking-control-2/)
**Published:** June 8, 2024
**Author:** Duckietown Admin
**Excerpt:** This Sim2Real study develops a vision-based reinforcement learning controller for Duckiebot lane following and collision avoidance.
**Content:**
##### General Information
- Vision-based Reinforcement Learning for Lane-Tracking Control
- András Kalapos, Csaba Gór, Róbert Moni, István Harmati
- Budapest University of Technology and Economics
- Kalapos, A., Gór, C., Moni, R. and Harmati, I., 2021. Vision-based reinforcement learning for lane-tracking control. Acta IMEKO, 10(3), pp.7-14.
[ Paper ](https://acta.imeko.org/index.php/acta-imeko/article/view/IMEKO-ACTA-10%20%282021%29-03-04)
[ GitHub ](https://github.com/kaland313/Duckietown-RL)
[ University ](https://www.bme.hu/en)
[ Authors ](#authors)
# Vision-based reinforcement learning for lane-tracking control
What is Vision-based Reinforcement Learning? A few important topics:
**Reinforcement Learning:** a machine learning paradigm where an agent learns to make decisions by interacting with an environment to achieve a goal. In this context, reinforcement learning is used to teach a vehicle how to drive within Duckietown lanes by providing rewards or penalties based on its actions.
**Vision-based Control:** The control of the vehicle is based on visual inputs, specifically images captured by a forward-facing camera. These images are processed by a neural network to determine appropriate steering actions, allowing the vehicle to track lanes and avoid collisions.
**Simulation-to-Reality (sim2real) Transfer Learning:** The trained policy, which learns to control the vehicle in a simulated environment, is transferred to real-world scenarios. The effectiveness of the trained model in real-world driving situations is evaluated, demonstrating the ability to generalize learning from simulation to reality.
**Domain Randomization:** This technique involves introducing variations or randomizations into the simulation environment during training. By exposing the agent to a wide range of simulated scenarios with different lighting conditions, road surfaces, and other environmental factors, domain randomization helps improve the model’s ability to generalize to unseen real-world conditions.
Learn about RL, navigation and other robot autonomy topics at the link below!
[ Duckietown Educational Resources ](https://duckietown.com/educational-resources/#machine-learning)
## Abstract
The present study focused on vision-based end-to-end reinforcement learning in relation to vehicle control problems such as lane following and collision avoidance. The controller policy presented in this paper is able to control a small-scale robot to follow the right-hand lane of a real two-lane road, although its training has only been carried out in a simulation.
This model, realised by a simple, convolutional network, relies on images of a forward-facing monocular camera and generates continuous actions that directly control the vehicle. To train this policy, proximal policy optimization was used, and to achieve the generalisation capability required for real performance, domain randomisation was used. A thorough analysis of the trained policy was conducted by measuring multiple performance metrics and comparing these to baselines that rely on other methods.
To assess the quality of the simulation-to-reality transfer learning process and the performance of the controller in the real world, simple metrics were measured on a real track and compared with results from a matching simulation. Further analysis was carried out by visualising salient object maps.
## Highlights - Vision-based reinforcement learning for lane-tracking control
Here is a visual tour of the work of the authors. For more details, check out the [full paper](#links "Post resources").
![Illustration of the policy architecture with the notations used.]()
Fig 1. Illustration of the policy architecture with the notations used.
![Explanation of the proposed orientation reward]()
Fig 2. Explanation of the proposed orientation reward
![Examples of domain randomised observations]()
Fig 3. Examples of domain randomised observations
![a) Test track used for simulated reinforcement learning and baseline evaluations; b) and c) real and simulated test track used for the evaluation of the simulation-to-reality transfer.]()
Fig 4. .a) Test track used for simulated reinforcement learning and baseline evaluations; b) and c) real and simulated test track used for the evaluation of the simulation-to-reality transfer.
![Learning curves for the reinforcement learning agent with different action representations and reward functions.]()
Fig 5. Learning curves for the reinforcement learning agent with different action representations and reward functions.
![Sequence of robot positions in a collision avoidance experiment with a policy trained using the modified orientation reward. After 𝑡=6 s, the controlled robot follows the vehicle in front at a short but safe distance until the end of the episode (approximate distance is calculated as the distance between the centre points of the robots minus the length of a robot).]()
Fig 6. Sequence of Robot Positions in Collision Avoidance Experiment with Modified Orientation Reward Trained Policy
![Salient objects highlighted on observations in different domains and tasks. Blue regions represent high activations throughout the network.]()
Fig 7. Salient objects highlighted on observations in different domains and tasks. Blue regions represent high activations throughout the network.
## Conclusion
Here are the conclusions from the authors of this paper:
“This work presented a solution to the problem of complex, vision-based lane following in the Duckietown environment using reinforcement learning to train an end-to-end steering policy capable of simulation-to-real transfer learning. It was found that the training is sensitive to problem formulation, such as the representation of actions.
This study has demonstrated that by using domain randomisation, a moderately detailed and accurate simulation is sufficient for training end-to-end lane-following agents that operate in a real environment. The performance of these agents was evaluated by comparing some basic metrics to match real and simulated scenarios.
Agents were also successfully trained to perform collision avoidance in addition to lane following. Finally, salient object visualisation was used to give an illustrative explanation of the inner workings of the policies in both the real and simulated domains.”.
## Project Authors
[András Kalapos](https://github.com/kaland313) is a Machine Learning PhD Student at [Budapest University of Technology and Economics](https://www.bme.hu/en), Hungary.
[Csaba Gór](https://www.linkedin.com/in/csaba-g%C3%B3r-a1964a109/?originalSubdomain=hu) is a Machine Learning Engineer at [Turbine](https://turbine.ai/), in Hungary.
[Róbert Moni](https://www.linkedin.com/in/robert-moni-942058100/?originalSubdomain=hu) is a Senior Machine Learning Engineer at [Continental](https://www.continental.com/en/).
[István Harmati](https://scholar.google.com/citations?user=6sgG0zkAAAAJ&hl=fr) is an Associate Professor at [Budapest University of Technology and Economics](https://www.bme.hu/en).
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Learn about Duckietown ](http://www.duckietown.com)
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**End-to-end Deep RL (DRL) systems:** in autonomous driving environments that rely on visual input for vehicle control face potential security risks, including:
- State Adversarial Perturbations: Subtle alterations to visual input that mislead the DRL agent, causing incorrect decision-making.
- Reward Tampering: Manipulation of the reward signal to misguide the learning process, leading the agent to adopt unsafe or inefficient policies.
These vulnerabilities can compromise the safety and reliability of self-driving vehicles.
**Categories:** paper, Research
**Tags:** AI, research
---
### [Evaluating Adversarial Robustness in Duckietown Navigation](https://duckietown.com/deep-rl-adversarial-robustness/)
**Published:** June 4, 2024
**Author:** Duckietown Admin
**Excerpt:** This study investigates security risks and evaluates adversarial robustness in DRL for autonomous driving in Duckietown's simulated environment.
**Content:**
##### General Information
- Deep Reinforcement Learning for Autonomous Navigation on Duckietown Platform: Evaluation of Adversarial Robustness
- Abdullah Hosseini, Saeid Houti, Junaid Qadir
- Qatar University
- A. Hosseini, S. Houti and J. Qadir, "Deep Reinforcement Learning for Autonomous Navigation on Duckietown Platform: Evaluation of Adversarial Robustness," 2023 International Symposium on Networks, Computers and Communications (ISNCC), Doha, Qatar, 2023, pp. 1-6, doi: 10.1109/ISNCC58260.2023.10323905.
[ Paper (ITSC) ](https://ieeexplore.ieee.org/document/10323905)
[ ResearchGate ](https://www.researchgate.net/publication/375968328_Deep_Reinforcement_Learning_for_Autonomous_Navigation_on_Duckietown_Platform_Evaluation_of_Adversarial_Robustness)
[ University ](https://www.qu.edu.qa/sites/)
[ Authors ](#authors)
# Deep RL for Autonomous Navigation on Duckietown Platform: Evaluation of Adversarial Robustness
What is adversarial robustness in navigation tasks all about? A few important topics:
**Reinforcement Learning (RL)** is a type of machine learning where agents learn to make decisions by receiving rewards or penalties based on their actions in an environment. This is great because it removed the need for curated training datasets.
**Deep Reinforcement Learning (DRL)** enhances RL by using deep neural networks to process complex inputs and make decisions. Deep networks are neural networks with multiple layers.
**Adversarial Robustness** refers to a system’s ability to resist and maintain performance despite deliberate attacks or input perturbations.
**Navigation** is the task of finding feasible paths between points in the environment like Google Maps or similar systems provide us in everyday life.
Learn about RL, navigation and other robot autonomy topics at the link below.
[ Duckietown Educational Resources ](https://duckietown.com/educational-resources/#machine-learning)
## Abstract
Self-driving cars have gained widespread attention in recent years due to their potential to revolutionize the transportation industry. However, their success critically depends on the ability of reinforcement learning (RL) algorithms to navigate complex environments safely. In this paper, we investigate the potential security risks associated with end-to-end deep RL (DRL) systems in autonomous driving environments that rely on visual input for vehicle control, using the open-source Duckietown platform for robotics and self-driving vehicles.
We demonstrate that current DRL algorithms are inherently susceptible to attacks by designing a general state adversarial perturbation and a reward tampering approach. Our strategy involves evaluating how attacks can manipulate the agent’s decision-making process and using this understanding to create a corrupted environment that can lead the agent towards low-performing policies. We introduce our state perturbation method, accompanied by empirical analysis and extensive evaluation, and then demonstrate a targeted attack using reward tampering that leads the agent to catastrophic situations.
Our experiments show that our attacks are effective in poisoning the learning of the agent when using the gradient-based Proximal Policy Optimization algorithm within the Duckietown environment. The results of this study are of interest to researchers and practitioners working in the field of autonomous driving, DRL, and computer security, and they can help inform the development of safer and more reliable autonomous driving systems.
## Highlights - Evaluation of Adversarial Robustness Results
Here is a visual tour of the work of the authors. For more details, check out the [paper link](https://ieeexplore.ieee.org/document/10323905).
![Fig. 1 - An illustration of the MDP framework, a fundamental concept in RL, where an agent interacts with an environment in a sequential manner to learn an optimal policy.]()
Fig. 1 - An illustration of the MDP framework, a fundamental concept in RL, where an agent interacts with an environment in a sequential manner to learn an optimal policy.
![Fig. 2 - High-level architecture for training a DRL agent.]()
Fig. 2 - High-level architecture for training a DRL agent.
![Average reward obtained by the agent for various reward functions in Duckietown]()
Fig. 3 - Average reward obtained by the agent for various reward functions.
![Average reward of the agent under two different reward functions. Increasing the value of ϵ resulted in a decrease in the agent’s distance traveled and position-orientation reward as it completely deviated from the roadable area and faced towards the wrong direction]()
Fig. 4 - Average reward of the agent under two different reward functions.
![Adversarial Navigation Robustness - Sequence of robot positions with DRL agent trained under adversarial and non-adversarial settings in a lane following experiment. The UAPFGSM method, making the agent move in circular movements with minimal perturbations, while adversarial reward tampering forces it to move in the opposite direction of the road.]()
Fig. 5 - Sequence of robot positions with DRL agent trained under adversarial and non-adversarial settings in a lane following experiment. The UAP- FGSM method, making the agent move in circular movements with minimal perturbations, while adversarial reward tampering forces it to move in the opposite direction of the road.
![Fig. 6 - Saliency map provides insight into this aspect of the model’s behavior, particularly in the context of adversarial attacks. Due to the universal perturbation of the UAP-FGSM attack, the agent’s attention to nearby yellow- dashed lines is reduced.]()
Fig. 6 - Saliency map provides insight into this aspect of the model’s behavior, particularly in the context of adversarial attacks. Due to the universal perturbation of the UAP-FGSM attack, the agent’s attention to nearby yellow- dashed lines is reduced.
## Conclusion
Here are the conclusions from the authors of this paper:
“The focus of our study was to address adversarial attacks on deep reinforcement learning (DRL) agents, specifically examining state adversarial attacks and reward-tampering attacks.
We developed a parametric framework for state adversarial attacks and a non-parametric framework for reward tampering attacks, which enabled us to create effective attacks. We found that the performance of a DRL agent declined rapidly after the attack, and the deviation from the road was worse than that of standard DRL.
We used salient maps to provide a clear explanation of the policies’ internal operations in both the adversarial and non-adversarial aspects. Our research provides insight into the potential vulnerabilities of DRL agents and highlights the need for more robust and secure agents to mitigate the risk of adversarial attacks.
Moving forward, future work will focus on incorporating real-world analysis to test the performance of the DuckieBot under both adversarial and non-adversarial settings”.
## Project Authors
[  ](https://www.linkedin.com/in/suri-rohit/)
[Abdullah Hosseini](https://www.linkedin.com/in/abdullah-hosseini-5aa77222b/?originalSubdomain=qa) is a Research and Development Specialist at [Weill Cornell Medicine](https://qatar-weill.cornell.edu/) in Qatar.
[  ](https://www.linkedin.com/in/suri-rohit/)
[Saeid Houti](https://www.linkedin.com/in/saeid-houti/?originalSubdomain=qa) is a Software Developer at [Ministry of Education and Higher Education Qatar](https://www.linkedin.com/company/qatar_edu/).
[  ](https://www.linkedin.com/in/suri-rohit/)
[Junaid Qadir](https://www.linkedin.com/in/junaidq/?originalSubdomain=qa) is a Professor of Computer Engineering at [Qatar University](https://www.qu.edu.qa/sites/).
### Learn more
Duckietown is a platform for creating and disseminating robotics and AI learning experiences.
It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.
[ Learn about Duckietown ](http://www.duckietown.com)
[ Get Started ](https://www.duckietown.com/guides)
[ Read more Papers ](https://www.duckietown.com/research/papers)
**End-to-end Deep RL (DRL) systems:** in autonomous driving environments that rely on visual input for vehicle control face potential security risks, including:
- State Adversarial Perturbations: Subtle alterations to visual input that mislead the DRL agent, causing incorrect decision-making.
- Reward Tampering: Manipulation of the reward signal to misguide the learning process, leading the agent to adopt unsafe or inefficient policies.
These vulnerabilities can compromise the safety and reliability of self-driving vehicles.
**Categories:** paper, Research
**Tags:** AI, research
---
### [Anatidaephilia: centralized city-based SLAM (cSLAM)](https://duckietown.com/anatidaephilia-centralized-city-based-slam-cslam/)
**Published:** April 26, 2024
**Author:** Duckietown Admin
**Excerpt:** Anatidaephilia is loving the idea that somewhere, somehow, a duck is watching you. The cSLAM equips Duckietowns with the ability to localize Duckiebots.
**Content:**
# Anatidaephilia: centralized city-based SLAM (cSLAM)
##### Project Resources
- **Motto**: Unum potest servare quod, carissimi mihi, anatis.
- **Objective**: Map the pose of the AprilTags and localize the positions of the Duckiebots as accurately as possible.
- **Authors**: Rohit Suri, Amaury Camus, Francesco Milano, Benson Kuan, Aleksander Petrov
[ Presentation ](https://hubs.ly/Q02v9dvj0)
[ Report ](https://docs-old.duckietown.org/daffy/opmanual_duckiebot/draft/demo_cslam.html)
[ Code ](https://github.com/duckietown/duckietown-cslam)
[ Authors ](#authors)
## cSLAM Project Description
**Project cSLAM** – Simultaneous Localization and Mapping (SLAM) is a successful approach for robots to estimate their position and orientation in the world they operate in, while at the same time creating a representation of their surroundings.
This project, centralized SLAM (or cSLAM), enables a Duckiebot to localize itself, while the watchtowers and Duckiebots work together to build a map of the city. The task is achieved by using the camera of the Duckiebot, together with watchtowers located along the path, to detect AprilTags attached to the tiles, the traffic signs, and the Duckiebot itself.
P. S. [Anatidaephilia](https://www.urbandictionary.com/define.php?term=Anatidaephilia "Anatidaephilia"), is Latin for loving, and being addicted to, the idea that somewhere, somehow, a duck is watching you.
## Project Highlights
Here is a visual tour of the work of the authors.
[Check out the documents](#links "project resources") for more details!
![Duckietown cSLAM algorithm physical architecture for localizing Duckiebots]()
cSLAM algorithm physical architecture for localizing Duckiebots. Watchtowers are traffic lights without lights, and are used to transform Duckietown in Autolabs.
![Duckietown cSLAM logical architecture]()
Duckietown cSLAM logical architecture for merging sensor measurements from robots and the city.
![Duckietown cSLAM graph optimizer representation]()
AprilTag detections from Watchtowers and Duckiebots are harmonized by solving a minimization problem.
![RViz visualization of Autolab virtual twin in Duckietown cSLAM project]()
RViz reconstruction of experimental localization outcomes.
![cSLAM Autolab watchtowers network diagnostic]()
cSLAM Autolab watchtowers network diagnostic
![Network nodes]()
## cSLAM Project Results
(Turn on the sound for best experience!)
This work developed into a paper, check the article [here](https://duckietown.com/integrated-benchmarking-and-design-for-reproducible-and-accessible-evaluation-of-robotic-agents/).
## Project Authors
[  ](https://www.linkedin.com/in/suri-rohit/)
[Rohit Suri](https://www.linkedin.com/in/suri-rohit/ "Rohit Suri") is a former Duckietown student, now a Roboticist at [Venti Technologies](https://ventitech.ai/ "Venti Technologies") in Singapore.
[Aleksandar Petrov](https://www.linkedin.com/in/aleksandar-petrov/ "Aleksandar Petrov") is a former Duckietown student, now a Ph. D. student at the [University of Oxford](https://www.ox.ac.uk/ "University of Oxford").
[Amaury Camus](https://www.linkedin.com/in/amaury-camus-ethz/ "Amaury Camus") is a former Duckietown student, now a Lead Robotics engineer at [Hydromea SA](https://www.hydromea.com/ "Hydromea SA") in Switzerland.
[Francesco Milano](https://www.linkedin.com/in/francesco-milano-ba5483132/ "Francesco Milano") is a former Duckietown student, now a Ph. D. student at [ETH Zurich](https://ethz.ch/en.html "ETH Zurich") in Switzerland.
[Benson Kuan](https://www.linkedin.com/in/benson-kuan-47a38992/ "Benson Kuan") is a former Duckietown student, now a Senior Robotics Research Engineer at [DSO National Laboratories](https://www.dso.org.sg/ "DSO National Laboratories") in Singapore.
### Learn more
Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ People of Duckietown interviews ](https://duckietown.com/news/people-of-duckietown/)
[ Get Started ](https://www.duckietown.com/guides)
[ More Projects ](https://www.duckietown.com/projects)
[ Get Robots ](https://get.duckietown.com/)
**Categories:** education, Projects
**Tags:** AI robotics, cSLAM, hands-on, localization, project, robot, robot autonomy, robotics, SLAM, smart city, smart-city
---
### [Dino Claro: a Duckietown journey from project to thesis](https://duckietown.com/dino-claro-duckietown-journey/)
**Published:** May 24, 2024
**Author:** Duckietown Admin
**Excerpt:** Dino Claro, a mechanical and mechatronics engineering graduate from the University of Cape Town, shares his Duckietown journey: with challenges and results.
**Content:**
# Dino Claro's Duckietown journey: from project to graduate thesis
Dino Claro, a mechanical and mechatronics engineering graduate from the University of Cape Town, shares his Duckietown journey: with challenges and results.
**Cape Town, February 13th, 2024:** Dino Claro, Graduate Mechanical and Mechatronics Engineer at the University of Cape Town, shares his experience with Duckietown and the project he developed using Duckiebots for his masters thesis.
##### Quick links
- [ Dino on Linkedin ](https://www.linkedin.com/in/dino-claro-5bb6981a0/)
- [ University of Cape Town ](https://uct.ac.za/)
- [ Agent code ](https://github.com/Dinoclaro/DT_objdet)
- [ YOLOv5 training dataset ](https://github.com/Dinoclaro/DT_objdet_train)
- [ Thesis ](https://drive.google.com/drive/folders/1ZwLF2rD6NRsYt3L-xwFWyfZLLXNE6PdJ)
## Duckinator: an Odometry pose-estimation for the Duckiebot robotic car platform
##### Hello and welcome Mr. Dino Claro! Could you introduce yourself?
My name is Dino Claro and I’m a Graduate Mechanical and Mechatronics Engineer at the University of Cape Town.
##### Thanks for accepting to share your experience with us. When did you first run into Duckietown?
During vacation work at the University of Cape Town (UCT) [Mechatronic Systems Group](https://ebe.uct.ac.za/department-electrical-engineering/research-groups/robotics), I was given the open-ended task of estimating the pose of a robot car. The goal of the vacation work was to solve a problem independently but also free from the
stresses of receiving a mark or grade. There was no expectation for novel work. In fact, the vacation work was only two weeks, and the expected solution would have been straightforward, probably odometry-based. Thus, Duckinator was born.
##### That's when you decided to use Duckietown?
With two platforms available, a basic Arduino 4WD kit and the Duckiebot, I could simply not resist the Duckies’ pull. The idea of using a Linux-based platform geared toward AI was extremely exciting.
At the end of the two-week vacation work, I was still ploughing through Duckietown documentation, the EdX: Self-Driving Cars with Duckietown MOOC, and ROS tutorials. My pose estimation solution seemed very far down the road. At that point, I should have realised that the DB (besides its cute exterior) is nuanced, to say the least.
##### Could you describe us your project?
The early phase of my project was extremely rudimentary. I had only had a couple of weeks during the vacation work to play with the DB \[[Duckiebot](https://get.duckietown.com/products/duckiebot-db21 "Duckiebot (DB21) online store")\]. I planned to continue with the EdX MOOC \[[Self-Driving Cars with Duckietown, 2023 edition](https://duckietown.com/mooc/ "Self-Driving Cars with Duckietown massive open online course (MOOC)")\] while researching Docker and ROS on the side for the first couple of weeks and then begin development. A pitfall with this technique was completing a section of the MOOC or some other tutorial and believing I could implement it myself. My initial thinking was that if the MOOC could be completed in 10 weeks or so and given that I have already a couple of weeks’ headstart due to vacation work, I should be able to implement my standalone autonomous solution for the DB in the 12-week frame.
Spoiler alert, Duckinator did not rival Tesla. I made the realisation about 4 weeks into the project. At that stage, I was in the Object Detection activity of the MOOC. With the world in a frenzy over AI and ML, I was itching to dip my hands in some of this mysterious ML stuff.
Dr. Pretorius obliged, and my plan from this point was to implement my own standalone Duckietown-compliant Docker image for the YOLOv5. Charged with the excitement of the new project direction, I began researching ML, computer vision algorithms and YOLO itself. Implementing the YOLOv5 model was relatively smooth sailing and I loved learning computer vision. In all honesty, my YOLOv5 model was just organising the Object Detection MOOC into a standalone Docker image as the MOOC hides the Docker image from the student. I obtained the training data using the MOOC helper files and then trained the YOLOv5 model using a very similar Google Colab script as provided by the MOOC.
I slightly extended the YOLOv5 model from the MOOC by training the model to detect DBs, which proved to be sort of successful. As I only had one Duckiebot, I tested the model by parking Duckinator in front of a mirror or putting it in front of my laptop showing photos of other DBs. Due to this shabby testing, I left this extension out of my write-up. This was all completed after week 7.
> With the world in a frenzy over AI and ML, I was itching to dip my hands in some of this mysterious ML stuff.
>
> Dino Claro
##### Did you meet your objectives?
Completing the Object Detection model effectively meant that my revised project brief had been met but as I still had some time, I needed to extend the model in some way.
Duckinator had eyes but I wanted to make it move … autonomously. I had the idea of creating a safety controller where the distance of objects from the duckiebot could be inferred using the predicted bound box and perspective geometry.
My theory went as follows: knowing the real-world size of all the objects the DB could detect and comparing this to the dimensions of the bounding box provided by the YOLO model, it would be possible to infer the depth of the object and this depth could then be used to base autonomous controller commands. This led me to research autonomous vehicle safety architectures/controllers and modern depth estimation algorithms. I soon realised that much more advanced autonomous architectures existed. For instance, modern autonomous vehicles fuse camera feeds, object detection models, kinematic models and various other sensors to generate vector or depth maps. The creation of these depth maps is extremely complex and a field of intense research.
##### What where the challenges you encountered during your project?
After coding up my algorithm for the projective projection algorithm, I obtained unexpected results. Negative in most cases. Describing my algorithm in more clarity to Dr. Pretorius, he made it clear that a simple projection perspective would not work in this case.
I was projecting everything from the camera image to the ground plane but of course, the duckies and any other objects do not exist solely on the ground plane. This being week 10 of the project, I had simply run out of time and had to scrap the projective perspective and had no time to implement any of the more complex algorithms out there.
I was devastated at the fact that Duckinator was not going to move.
Upon some reflection though, the YOLOv5 model was working quite well, and I had all this research about autonomous architectures and depth estimation. One of the autonomous architectures I researched was [Braitenberg vehicles acting as, possibly, the simplest autonomous architecture](https://github.com/duckietown/duckietown-lx/tree/mooc2022/braitenberg "Duckietown Braitenberg Vehicles learning experience"). A basic Braitenberg controller was simple enough to implement and would mean once again Duckinator could move.
Bounding boxes were populated onto a black image and then divided into left and right region maps. These maps were then element-wise multiplied with a weight matrix to provide a scalar value which can be used for wheel commands. Using the ‘fear’ Braitenberg vehicle the DB would then steer away from any detected objects. Another realisation was that my project was experimental with one of the main goals being for it to act as a stepping stone for future projects.
At this stage (two weeks before my project was due) I was satisfied with a newly engineered aim: Evaluating the viability of the Duckietown platform at the undergraduate level by implementing an ML object detection model. The key outputs being the YOLOv5 model (Duckie Detector) as well as possible future projects and trajectories for students.
> The real learning occurs when getting your hands dirty, experimenting and troubleshooting.
>
> Dino Claro
##### What are your final considerations?
Reflecting on my journey from a complete beginner to a slightly more competent beginner, here’s my advice for those on a similar journey:
Begin with a blank Duckietown-compliant Docker image and dive into coding a demo, whether it’s based on my solution or another. Ultimately, the goal is to first understand the code and then attempt to recreate it without directly copying.
While documentation and EdX activities are useful in providing broad overviews and points of contact for debugging, relying solely on them may create a deceptive sense of competency.
The real learning occurs when getting your hands dirty, experimenting and troubleshooting.
##### Thank you very much for taking the time, we appreciated your story very much! Is there anything else you would like to add?
Yes, I would say that embracing the hands-on experience is key to understanding the platform and being immersed in the infectious ethos surrounding Duckietown.
On that note, I would like to express my gratitude to [Dr. Pretorius](https://ebe.uct.ac.za/department-mechanical-engineering/contacts/arnold-pretorius "Arnold Pretorius") for granting me the freedom to experiment and the opportunity to work with the Duckiebot. I eagerly await future projects and the growth of the Duckietown community
### Learn more about Duckietown
Duckietown enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** News, People
**Tags:** student
---
### [Massachusettes Institute of Technology: first BCI hackathon](https://duckietown.com/mit-bci-hackathon/)
**Published:** May 20, 2024
**Author:** Duckietown Admin
**Excerpt:** Take a look at MIT's first BCI hackathon, and how students used their brains to move around Duckiebots in Duckietowns filled with pedestrian-obstacle duckies.
**Content:**
**Cambridge, MA, USA – McGovern Institute**, 24-25 February 2024: Over 100 participants took part to the first MIT BCI hackathon, competing in teams to control Duckiebots using brain computer interfaces.
##### Quick links
- [ Massachusetts Institute of Technology ](https://www.mit.edu/)
- [ Brain Computer Initiative ](https://bci-i.github.io/)
- [ MIT-BCI hackathon ](https://bci-i.github.io/#/hackathon)
- [ Federico Claudi ](https://www.linkedin.com/in/federico-claudi-3b31b5262/)
## Controlling Duckiebots using brain computer interfaces
Over 100 participants gathered at the Massachusetts Institute of Technology for the first BCI hackathon organized by Dr. Federico Claudi. The participants tried to control a Duckiebot using only brain computer interfaces, and competed in a series of tasks.
BCI is the field of research that studies how to measure, amplify, filter and utilize electrical signals from the brain to interact with external devices.
What made this hackathon distinctive was the hands-on challenge, where participants were tasked with controlling a physical robot. This not only tested participants’ technical skills but also showcased their ability to tackle real-world problems through innovative BCI applications.
The task teams competed on was having Duckiebots (DB21-J4) navigate a road loop as fast as possible while avoiding Duckies. Here is an example:
The hardware used in this competition was an [X.on EEG](https://xon-eeg.com/) headset, and Duckiebots for control. Also, the winning team’s solution will be soon made available as a reproducible Learning Experience with Duckietown – stay tuned!
The Duckiebot is a DIY, Raspberry Pi-based robot powered by Nvidia and designed for introducing learners to autonomous technologies.
[ Get the Duckiebot ](https://get.duckietown.com/products/duckiebot-db21)
[  ](https://get.duckietown.com/products/duckiebot-db21)
If you would like to contribute in developing accessible BCI LXs with Duckietown, and support the dissemination of BCI research, e.g. reach out to us at .
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
[ Request a quote for your class ](https://contact.duckietown.com/request-a-quote)
**Categories:** Blog, editor-choiche, Events, News
---
### [Round 2 of the the AI Driving Olympics is underway!](https://duckietown.com/round-2-of-the-the-ai-driving-olympics-is-underway/)
**Published:** April 15, 2019
**Author:** Jacopo Tani
**Content:**
The AI-DO is back!
We are excited to announce that we are now ready to **accept submissions** for AI-DO 2, which will culminate in a live competition event to be held at ICRA 2019 this May 20-22.
The AI Driving Olympics is a global robotics competition that comprises a series of challenges based on autonomous driving. The AI-DO provides a standardized simulation and robotics platform that people from around the world use to engage in friendly competition, while simultaneously advancing the field of robotics and AI.
Check out our official [press release](https://www.duckietown.com/archives/35882).
 The finals of AI-DO 1 at NeurIPS, December 2018
We want to see your classical robotic and machine learning based algorithms go head to head on the competition track. Get started today!
Want to learn more or join the competition? Information and get started instructions are [here](https://www.duckietown.com/research/AI-Driving-Olympics).
#### If you've already joined the competition we want to hear from you!
Share your pictures on [facebook](https://www.facebook.com/duckietown/) and [twitter](https://twitter.com/DuckietownAI).
Get involved in the community by:
**asking for help**
- File a [github issue](https://github.com/duckietown)
- [Ask a question](http://duckietown.com/questions)
- Join the [slack community](https://join.slack.com/t/duckietown/shared_invite/enQtNTU0Njk4NzU2NTY1LTQ2MDI4MTY1OTE1YjhjMTU4YTdkMDViMzJmNmJkNGQxN2U1ZGJlZjk2NGM0M2FiODY3YmQ2MTQ3MGM2MjY1ZTI) and discuss
**offering help**
- Solve a [github issue](https://github.com/duckietown)
- [Answer questions](http://duckietown.com/questions)
- Offer support in [slack](https://join.slack.com/t/duckietown/shared_invite/enQtNTU0Njk4NzU2NTY1LTQ2MDI4MTY1OTE1YjhjMTU4YTdkMDViMzJmNmJkNGQxN2U1ZGJlZjk2NGM0M2FiODY3YmQ2MTQ3MGM2MjY1ZTI) discussions
**Categories:** Blog, Events, News
**Tags:** AI Driving Olympics, AI-DO, announcement, competition, global, icra, montreal
---
### [The Duckietown path to robotics: an interview with Prof. Francesco Maurelli](https://duckietown.com/the-duckietown-path-to-robotics-an-interview-with-prof-francesco-maurelli/)
**Published:** July 4, 2022
**Author:** Ivano Marocchi
**Content:**
**Jacobs University, Bremen, June 1, 2022**: Francesco Maurelli, professor at the Jacobs University of Bremen, talks about how Duckietown impacted his work and his academic career.
## The Duckietown path to robotics: an interview with Prof. Francesco Maurelli
Francesco Maurelli, professor at the Jacobs University of Bremen, Germany, shares in the interview below his interaction with Duckietown.
##### Quick links
- [ Jacobs University, Bremen ](https://www.jacobs-university.de/)
- [ Marine Systems Jacobs University ](https://marine.jacobs-university.de/?_ga=2.24067066.1918914472.1654009364-815127560.1653405179)
**Let’s start by simply asking your name, who you are, where you work, what you do for a living.**
Hi, Federico. Hi, everyone. I am Francesco Maurelli and I’m a professor in robotics at Jacobs University in Bremen.
**When was the first time you came across Duckietown in your life? Describe your first contact with Duckietown for us.**
Well, that was in my team. I was there as a Marie Curie scholar from Europe. I met Andrea and Liam, and learned about this new initiative. I was interested so I spoke in depth with them and with the students who took the course. I then looked at the videos and thought it was a great setup because it brings robotics closer to the students in a fun way by reducing the access barrier. Many people think that robotics is very hard, which is true. I’m not saying it’s easy, but on the other hand, there are easier paths to access robotics. Additionally the element of gamification makes people happy when they work with Duckietown. I found that students want to get involved regardless of the course work, they just like the concept.
**Thank you. Is there a specific thing that maybe you did using Duckietown in your life, a project, a program or some sort of ecosystem?**
We have had three different initiatives based on Duckietown.
The first one, called Jacobs Robotics, was an extracurricular activity for students. I would meet interested students outside of class time, it wasn’t linked to academic credit. It was just for fun and for learning. Among the different robotics platforms, we had a group working on Duckietown. This was the initial step in using Duckietown at our university.
Then the second step was to embed Duckietown in the official curriculum. We have a bachelor’s program in robotics and intelligence systems, and I’m its program manager. We were rewriting and updating some parts of it as we underwent a new wave of accreditation. So I took the opportunity to redesign some aspects of the program and in this process decided to embed Duckietown at Bachelor level. I’ve introduced it at Ross Lab in simulation in the fall of the second year, in the third semester, and then we have a robotics project based on Duckietown in the spring, ofthe second year, (i.e. in the fourth semester). That means that when students start their third year, they already have an understanding of ROS, they have knowledge of Duckietown and they work with real systems. This means that they can do a much better thesis, even if it’s a Bachelor level, we can improve the average level. When I joined the University, the first month of the thesis was lost on students learning to use Ross, for example. Now we are a step ahead.
The third part is the research application. It’s not only a matter of having fun with students, or delivering courses to students, but also doing my own research. I have a project which is funded by the German Research Foundation, DFG, it is in collaboration with the psychology Department. The psychologists want to look at the characteristics that humans assign to entities to identify the “self”. We as roboticists are going to develop and program different robotic behaviors, which the behavioral scientist from the psychology department will analyze. In a nutshell, we will prepare different videos illustrating the same actions performed in different ways. A very basic example would be moving in a city at constant speed or moving in a city at variable speeds. Our partners in the psychology department will show these videos to the study participants and collect user feedback through questionnaires to determine which behaviors they think are more alike a self determined behavior.
**This is extremely interesting. Thank you very much. Have you ever heard about the MOOC course?**
Yes, actually. In fact I suggested to our students to look at the MOOC. Of course, it is set up in a different way with respect to our course, but it can be and it has been a useful tool for students to review some of the material through a different viewpoint. So it’s definitely a valuable learning material which is available to the community.
> "It's not only a matter of having fun with students, or delivering courses to students, but also doing my own research."
>
> Francesco Maurelli
**Okay. My last question is would you suggest Duckietown to other people, colleagues or your students? And why?**
Absolutely. I see that from my own experience, students like it and they learn a lot about robotics. All the different concepts ranging from control to localization to computer vision can be applied in Duckietown. So in our projects, in our robotics projects, different groups of students develop different ideas. And I see that they are enjoying themselves and they are learning. So it’s definitely a plus.
**Thank you very much.**
### Learn more about Duckietown
The Duckietown platform offers robotics and AI learning experiences.
Duckietown is modular, customizable and state-of-the-art. It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Find out more use cases ](https://www.duckietown.com/news/people-of-duckietown)
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** People
---
### [Self-driving cars, meet rubber duckies - MIT News](https://duckietown.com/self-driving-cars-meet-rubber-duckies-mit-news/)
**Published:** April 20, 2016
**Author:** Duckietown Admin
**Excerpt:** Self-driving cars, meet rubber duckies MIT News
**Content:**
[Self-driving cars, meet rubber duckies](https://news.mit.edu/2016/self-driving-cars-meet-rubber-duckies-0420) MIT News
**Categories:** Google News
---
### [STEM Intensive Learning with Prof. Krinkin](https://duckietown.com/community-spotlight-kirill-krinkin-stem-intensive-learning-approach/)
**Published:** November 18, 2019
**Author:** Duckietown Admin
**Content:**
In the world of engineering education, there are many excellent courses, but often the curriculum has one serious drawback – the lack of good connectivity between different topics. Over in Saint Petersburg, Russia, [Kirill Krinkin](https://kirill.krinkin.com/) from [SPbETU](https://etu.ru/) and[ JetBrains Research](https://research.jetbrains.org/) has been using Duckietown to address this problem through an intensive STEM winter course.
## STEM Intensive Learning Approach
by Kirill Krinkin
The first part of the school program was a week of classes in the base topic areas which were chosen to complement each other and help students see the connection between seemingly different things – mathematics, electronics and programming.
Of course, the main goal of the program was to give students the opportunity to put their new found knowledge into practice themselves.
Duckietown was the perfect fit for our course because it offered a hands-on learning experience for all of our main topics areas, and once we covered those subject in the first lessons, we challenged the students with much more complex tasks – in the form of projects – in the second half of the course. It made for an exciting and engaging curriculum because students could address a problem, write a program to solve it, and then immediately launch it on a real robot.
The main advantage of Duckietown compared to many other platforms is that there is a very small learning curve: people who knew nothing about programming and robotics started working on projects after only a few days!
Overview of the course
Part 1 – Main Topic Areas
*Subject 1: Linear Algebra*
Students spent one day studying vectors and matrices, systems of linear equations, etc. Practical tasks were built in an interactive mode: the proposed tasks were solved individually, and the teacher and other students gave comments and tips.
*Subject 2: Electricity and Simple Circuits*
Students studied the basics of electrodynamics: voltage, current, resistance, Ohm’s law and Kirchhoff’s laws. Practical tasks were partially done in the electric circuits simulator or performed on the board, but more time was devoted to building real circuits, such as logic circuits, oscillatory circuits, etc.
*Subject 3: Computer Architecture*
In a sense, a bridge connecting physics and programming. Students studied the fundamental basis, the significance of which is more theoretical than practical. As a practice, students independently designed arithmetic-logic circuits in the simulator.
*Subject 4: Programming*
Python 2 was chosen as the programming language, as it is used in programming under ROS. After we taught the material and gave examples of solving problems, students were challenged with their own problems to solve, which we then evaluated.
*Subject 5: ROS*
Here the students started programming robots. Throughout the school day, students sat at computers, running the program code that the teacher talked about. They were able to independently launch the basic units of ROS, and also get acquainted with the Duckietown project. At the end of this day, students were ready to begin the design part of the course – solving practical problems.
Part 2 – Projects
*1. Calibration of colors*
Duckiebots needs to calibrate the camera when lighting conditions change, so this project focussed on the task of automatic calibration. The problem is that color ranges are very sensitive to light. Participants implemented a utility that would highlight the desired colors on the frame (red, white and yellow) and build ranges for each of the colors in HSV format.
*2. Duck Taxi*
The idea of this project was that Duckiebot could stop near some object, pick it up and then continue along, following a certain route. Of course, a bright yellow Duckie was the chosen passenger. The participants divided this task into two: detection and movement along the graph.
drive while Duckie is not detected
Duckie identified as a yellow spot with an orange triangle 🙂
Building a route according to the road graph and destination point
*3. Building a road map*
The goal of this project was to build a road map without providing a priori environmental data for the Duckiebot, relying solely on camera data. Here’s the working scheme of the algorithm developed by the participants:
*4. The patrol car*
This project was invented by the students themselves. They offered to teach one Duckiebot, the “patrol”, to find, follow, and stop an “intruding” Duckiebot. The students used ArUco markers to identify the Intruder on the road as they are easy to work with and they allow you to determine the orientation and distance of the marker. Next, the team changed the state machine of the Patrol Duckiebot so that when approaching the stop-line the bot would continue through the intersection without stopping. Finally, the team was able to get the Patrol Duckiebot to stop the Intruder bot by connecting via SSH and turning it off. The algorithm of the patrol robot can be represented as the following scheme:
**Summary**
Students walked away from our STEM intensive learning program with the foundations of autonomous driving, from the theoretical math and physics behind the programming and circuitry to the complex challenges of navigating through a city. We were successful in remaining accessible to beginners in a particular area, but also providing materials for repetition and consolidation to experienced students. Duckietown is an excellent resource for bringing education to life.
After our course ended students were asked about their experience. 100% of them said that the program exceed their expectations. We can certainly say that the Duckietown platform played a pivotal role in our success.
**Categories:** education, News, People
**Tags:** news, russia, stem, world
---
### [Learn to Program Self-Driving Cars With Duckietown - IEEE Spectrum](https://duckietown.com/learn-to-program-self-driving-cars-and-help-duckies-commute-with-duckietown-ieee-spectrum/)
**Published:** August 20, 2018
**Author:** Duckietown Admin
**Excerpt:** Learn to Program Self-Driving Cars (and Help Duckies Commute) With Duckietown IEEE Spectrum
**Content:**
[Learn to Program Self-Driving Cars (and Help Duckies Commute) With Duckietown](https://spectrum.ieee.org/automaton/robotics/artificial-intelligence/learn-to-program-self-driving-cars-and-help-duckies-commute-with-duckietown) IEEE Spectrum
**Categories:** Google News
---
### [Monocular Robot Navigation with Self-Supervised Pretrained Vision Transformers](https://duckietown.com/monocular-robot-navigation-with-self-supervised-pretrained-vision-transformers/)
**Published:** May 9, 2022
**Author:** Ivano Marocchi
**Content:**
# Monocular Robot Navigation with Self-Supervised Pre-trained Vision Transformers
Duckietown’s infrastructure is used by researchers worldwide to push the boundaries of knowledge. Of the many outstanding works published, today we’d like to highlight “Monocular Robot Navigation with Self-Supervised Pretrained Vision Transformers” by Saavedra-Ruiz et al. at the University of Montreal.
Using visual transformers (ViT) for understanding their surroundings, Duckiebots are made capable of detecting and avoiding obstacles, while safely driving inside lanes. ViT is an emerging machine vision technique that has its root in Natural Language Processing (NLP) applications. The use of this architecture is recent and promising in Computer Vision. Enjoy the read and don’t forget to reproduce these results on your Duckiebots!
## Abstract
“In this work, we consider the problem of learning a perception model for monocular robot navigation using few annotated images. Using a Vision Transformer (ViT) pretrained with a label-free self-supervised method, we successfully train a coarse image segmentation model for the Duckietown environment using 70 training images. Our model performs coarse image segmentation at the 8×8 patch level, and the inference resolution can be adjusted to balance prediction granularity and real-time perception constraints. We study how best to adapt a ViT to our task and environment, and find that some lightweight architectures can yield good single-image segmentations at a usable frame rate, even on CPU. The resulting perception model is used as the backbone for a simple yet robust visual servoing agent, which we deploy on a differential drive mobile robot to perform two tasks: lane following and obstacle avoidance.”
##### Authors
- [ Miguel Saavedra-Ruiz ](https://mikes96.github.io/)
- [ Sacha Morin ](https://sachamorin.github.io/dino/)
- [ Liam Paull ](https://liampaull.ca/)
- [ Paper ](https://arxiv.org/abs/2203.03682)
- [ Code ](https://github.com/sachaMorin/dino)
### Pipeline
“We propose to train a classifier to predict labels for every 8×8 patch in an image. Our classifier is a fully-connected network which we apply over ViT patch encodings to predict a coarse segmentation mask:”
### Conclusions
“In this work, we study how embodied agents with visionbased motion can benefit from ViTs pretrained via SSL methods. Specifically, we train a perception model with only 70 images to navigate a real robot in two monocular visual-servoing tasks. Additionally, in contrast to previous SSL literature for general computer vision tasks, our agent appears to benefit more from small high-throughput models rather than large high-capacity ones. We demonstrate how ViT architectures can flexibly adapt their inference resolution based on available resources, and how they can be used in robotic application depending on the precision needed by the embodied agent. Our approach is based on predicting labels for 8×8 image patches, and is not well-suited for predicting high-resolution segmentation masks, in which case an encoder-decoder architecture should be preferred. The low resolution of our predictions does not seem to hinder navigation performance however, and we foresee as an interesting research direction how those high-throughput low-resolution predictions affect safety-critical applications. Moreover, training perception models in an SSL fashion on sensory data from the robot itself rather than generic image datasets (e.g., ImageNet) appears to be a promising research avenue, and is likely to yield visual representations that are better adapted to downstream visual servoing applications.”
### Learn more
The Duckietown platform offers robotics and AI learning experiences.
Duckietown is modular, customizable and state-of-the-art. It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Find out more ](http://www.google.com)
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
**Categories:** paper, Research
---
### [Spearheading Autonomy: Giulio Vaccari and the amazing AEA](https://duckietown.com/duckietown-politecnico-milano-giulio-vaccari/)
**Published:** January 25, 2024
**Author:** Duckietown Admin
**Excerpt:** Giulio Vaccari tells us how he spearheaded Duckietown at the Milan Polytechnic through the AEA student association, to support autonomy research and networking.
**Content:**
Giulio Vaccari tells us how he spearheaded Duckietown at the Milan Polytechnic through the AEA student association, to support autonomy research and networking.
**Milan, February 29th:** Giulio Vaccari, master student at the time of this interview, tells us how and why he spearheaded the use of Duckietown at the Polytechnic of Milan, one of the most important Italian engineering universities.
##### Quick links
- [ Giulio on Linkedin ](https://www.linkedin.com/in/giuliovaccari/)
- [ Politecnico di Milano (PoliMi) ](https://www.polimi.it/)
- [ Giulio Vaccari ](https://giuliovaccari.it/)
- [ Automation Engineering Association ](https://www.aeapolimi.it/en/)
# Spearheading Autonomy: Giulio Vaccari and the amazing student Automation Engineering Association (AEA)
## An interview with Giulio Vaccari from the Polytechnic University of Milan
##### Hello and welcome Mr. Giulio Vaccari! Could you introduce yourself?
My name is Giulio Vaccari, and I’m a master student at the Politecnico di Milano (“Polimi”). I am studying automation and control engineering and I am right now in the process of finishing my thesis with Duckietown. During my five years here, I founded the student Automation Engineering Association, focused on automation engineering. At the “Polimi” the course of control and automation engineer is not super big, now it’s getting bigger. More and more people are joining, but it’s still pretty small compared to, for example, computer science or other bigger courses. The idea of our association was to help with networking, help to understand what are future possible career paths as automation engineers. That’s why together with some friends we decided to found the association some years back.
##### Very interesting. How does Duckietown fit in all this?
It all started when I met and talked with Jacopo Tani and Vincenzo Polizzi I believe three years ago.
I had some friends, older friends, at ETH Zurich that introduced us to Duckietown as a possible project for our association, that at the time was still pretty small. We were newborn and we liked the idea of start working with Duckietown right away, because we were looking for a project that we could treat like a competition. It’s easier to get more people to know each other and network, when they have a common goal, as there are common topics to discuss.
I wanted something that would help us develop teamwork skills, something around which to build a strong team. Then this idea evolved a little bit and now we have a very focused and skilled team that works with aim of doing good at the [Artificial Intelligence Driving Olympics (AI-DO)](https://duckietown.com/research/ai-driving-olympics/ "Artificial Intelligence Driving Olympics (AI-DO)"). We did our first competition in 2021, in December, and it went well. Now we are improving. We are still working on it.
##### Could you tell us more about your team's experience with the AI-DO competition?
Sure! We ended up being finalist, I’m very proud of that! In the meantime, I talked with some professors to try to get some help to improve what we were doing with Duckietown, to have some suggestions on what we should focus on. I was lucky enough to meet a Professor, now my thesis advisor, who had already heard about Duckietown and liked it. I was very happy that we were able to bring Duckietown to the Politecnico of Milano. Although the AI-DO are over now, I’m still doing related work, so I’m able to help the team do better while working on my thesis. Duckietown is now part of the laboratory for autonomous driving here at PoliMi, and I feel like managing to get Duckietown to be adopted by the Politecnico di Milano is a pretty big thing that I’m still proud of!
##### And you should be! Do you use Duckietown also in the activities of the association?
When it comes to the association, we are focused on AI. We have a team of around 20 people working on different things. We use Duckietown also to do outreach when we have, for example, “associations days”, which are open days when students from high schools come and visit the University. That’s when we showcase Duckietown to show both what we do as an association but also what a control engineer is supposed to do, because it’s very difficult to explain otherwise.
##### What do you like about Duckietown? Why do you think it is useful for what you do?
Through Duckietown, it’s very easy to implement ideas in the real world. Usually if you need to build something like that from scratch, it will be super difficult. There are a lot of failure points, while with Duckietown we can just deploy it and it just works. This is super cool. We can also test a lot of different control strategies with very low effort. I mean, just the fact of writing the control algorithm, deploying it and it just works is huge. So this makes things super easy. Duckietown is very friendly and easy to use. It’s not something you often see in robotics, which is instead usually very complicated and intimidating. Yes, very intimidating. And Duckietown is even very friendly looking, and this is definitely a big plus!
##### Do you think people that are using Duckietown, are satisfied with it?
Yes, we had really positive feedback. I mean, everyone is really happy to work with Duckietown and we had so many people applying that we had to limit the team size because that would’ve made it very difficult to manage all those people. So I would say it had a very positive impact in general on the association. People are generally enthusiastic to just see the robots going around.
> There are a lot of failure points in robotics, while with Duckietown we deploy our algorithms and it just works. This is super cool. We can test a lot of different control strategies with very low effort.
>
> Giulio Vaccari
##### Would you suggest it to colleagues or other student associations?
Yes, for sure. But I see that it’s not only our team; also the Professor that I’m working with is very happy to be able to use it, also as a benchmark for different controls. There are other universities that have their very specific tracks to run their robots, while with Duckietown, it’s super easy to test like, the same strategy in different environments, but still with the same basics. This is a very positive thing also for the university.
##### Thank you very much for taking the time, we appreciated your story very much! Is there anything else you would like to add?
Right now I’m working on a different subject. So I’m using still Duckietown, but not in a city environment, but in a racing environment. My objective is to set up a racing track and have two robots compete against each other. They are still not competing, but we are getting close. It’s very nice that together with the Duckietown team, we are working on the urban environment, using different kind of materials to create these racing environments. I think it’s very cool.
There are also some issues every now and then. Maybe something that is not super easy to find, something that is very in depth, some APIs. If you want to use them, you need to dive deep. There is a support Slack channel though, and it’s super easy to ask for help. This too I believe is very nice and just adds to the positive experience.
## Special Giulio Vaccari content!
We found in our archives the video submission of Giulio Vaccari and his team at AEA on the occasion of the 2021 AI Driving Olympics finals. In this fun, short, video team PoliMi explains the rationale and challenges faced in developing an ML-based agent for the 2021 competition!
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences. It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** News, People
**Tags:** AEA, AI Driving Olympics, AI-DO, Duckiebots, student, student association
---
### [How Duckietown inspired a 14 year old girl to become a tech entrepreneur](https://duckietown.com/valeria-cagninas-experience-with-duckietown/)
**Published:** August 6, 2018
**Author:** Jacopo Tani
**Excerpt:** Learn how Duckietown inspired 14 year old Valeria Cagnina to pursue a career in robotics!
**Content:**
*We host a guest post by [Valeria Cagnina](https://valeriacagnina.tech/en/), who had the luck to meet our team very early – in fact, when the first Duckietown was still being built – and she helped with the tape!*
Nothing is impossible…the word itself says “I’m possible”!
I discovered robotics when I was 11 years old with a digital plant made with Arduino that I saw in Milan Coderdojo. I really liked robotics and decided I would like to make my own robot.
So I searched online for a robot I could make myself. I found some videos on the web about a robot from MIT. I really loved this wonderful robot… but I was too young and I didn’t have the skills necessary to build it. So I surfed online to search other types that would be easier to build, but in my mind remained the dream to go to see this cool robot at MIT in Boston.
After a while, following and making my own Youtube videos, I made my first robot alone at 11 years old: it could move itself around a room avoiding obstacles thanks to its distance sensor programming with Arduino.
In Italy it was not so common to make a robot at 11, so I was able to share this experience a lot of events and conferences that brought me to speak in a TEDx at 14 years old.
Casually, at the same age, I travelled to United States to visit New York, Boston and Canada… at the beginning it seemed a normal holiday…
I convinced my parents to extend our trip to stay more time around MIT. We went sightseeing in Boston and in MIT but it wasn’t enough for me! I wanted to look inside this place that was so magical to me, and I especially wanted to talk with the engineers that build and program robots! Maybe I would see that same robot that I found when I was 11 years old!
 The early stages of Duckietown at MIT
I left my parents visiting the rest of Boston and I started to go alone around the MIT departments, trying to open every door that I found in front of me.
While I was walking, I was looking through the laboratories windows and my attention was caught by an empty room -I mean with no humans inside 😀 ! – full of duckies and with a sort of track for cars on the floor.
What was this room about? What was the purpose of these duckies? I was very, very curious about it and had many questions, but there was no one in the lab!
Obviously I never give up, I absolutely believe that nothing is impossible so, every day, until my departure to the next leg of our trip, I continued to go around MIT passing in front of THAT lab hoping to find someone in it.
Finally one day I saw some people inside the lab doing something. I was really excited! I watched them from the window. I absolutely wanted to know what they were doing – one of them was soldering, another one was using duct tape. Suddenly they saw me and they invited into the lab! What an astonishment for me!
Immediately they asked me a lot of questions: why was a 14 year old roaming MIT alone, why was I so excited about that lab… Then one of them (I didn’t know his name) asked if I wanted to help build “Duckietown”. He told me about the project (at that time it wasn’t started yet) and he asked me about myself and the first robot I built. After an afternoon spent together, I discovered that this strange guy was Andrea Censi, one of the founders of the Duckietown project! Amazing!
Andrea proposed to me a challenge: I had to try to make my own Duckietown robot, a Duckiebot. Since it was a university project, I was able to follow the online tutorials and ask lots of questions to all the other Duckietown members on the communication forum, Slack. He had only one request of me: he told me that even though the robot was hard to build and program, I shouldn’t give up.
I was so happy that I immediately agreed. I was handed the robot kit, a list of various links and some Duckies .
Now it was my turn! I didn’t want to disappoint Andrea, so as soon as I arrived in Italy I put myself to work but, wow, building the Duckiebot was very hard! I spent an entire afternoon trying to comprehend just 4 rows of the tutorial. I began to ask questions on Slack and I tried, I tried and I tried again.
I never worked with Linux before so that was a completely new world for me. I started from the beginning, without knowledge at all but I worked for a few months until I received a message from Andrea: “Do you want to spend some time here, in Boston, working with us in Duckietown?” Of course I was willing, I couldn’t wait, it was an amazing proposal!
So I became a Duckietown Senior Tester at 15 years old and I spent almost all the summer inside the labs of MIT. My task was simplifying the university-level tutorial and making it accessible to the high-school students (like me ) as well as making the Duckiebot, which had now evolved!
Thanks to the help of Andrea and Liam (the other founder) I finally succeeded to program my robot: it was now able to drive autonomously in Duckietown. If felt like a dream come true!
Spending the summer in Duckietown at MIT allowed to me to discover a completely new world: I understood that education could be playful and that learning could be fun!
 Valeria's duckiebot (back)
 Valeria's Duckiebot (side)
**Categories:** outreach, People
**Tags:** independent, Italy, makademia, maker, outreach, world
---
### [Duckietown: An Innovative Way to Teach Autonomy](https://duckietown.com/learning-autonomous-systems-an-interdisciplinary-project-based-experience-6/)
**Published:** June 5, 2016
**Author:** Ivano Marocchi
**Content:**
- [ Title: Duckietown: An Innovative Way to Teach Autonomy ](https://link.springer.com/chapter/10.1007/978-3-319-55553-9_8)
- Authors: Jacopo Tani, Liam Paull, Maria T. Zuber, Daniela Rus, Jonathan How, John Leonard, Andrea Censi
- Published in EDURobotics 2016
## Duckietown: An Innovative Way to Teach Autonomy
Teaching robotics is challenging because it is a multidisciplinary, rapidly evolving and experimental discipline that integrates cutting-edge hardware and software. This paper describes the course design and first implementation of Duckietown, a vehicle autonomy class that experiments with teaching innovations in addition to leveraging modern educational theory for improving student learning. We provide a robot to every student, thanks to a minimalist platform design, to maximize active learning; and introduce a role-play aspect to increase team spirit, by modeling the entire class as a fictional start-up (Duckietown Engineering Co.). The course formulation leverages backward design by formalizing intended learning outcomes (ILOs) enabling students to appreciate the challenges of: (a) heterogeneous disciplines converging in the design of a minimal self-driving car, (b) integrating subsystems to create complex system behaviors, and (c) allocating constrained computational resources. Students learn how to assemble, program, test and operate a self-driving car (Duckiebot) in a model urban environment (Duckietown), as well as how to implement and document new features in the system. Traditional course assessment tools are complemented by a full scale demonstration to the general public. The “duckie” theme was chosen to give a gender-neutral, friendly identity to the robots so as to improve student involvement and outreach possibilities. All of the teaching materials and code is released online in the hope that other institutions will adopt the platform and continue to evolve and improve it, so to keep pace with the fast evolution of the field.
### Did you find this interesting?
Read more Duckietown based papers [here](https://www.duckietown.com/research/papers).
**Categories:** paper, Research
---
### [Global Demo Day 2017 at Montreal, Zurich and Chicago](https://duckietown.com/global-demo-day-2017-at-montreal-zurich-and-chicago/)
**Published:** January 20, 2018
**Author:** Duckietown Admin
**Content:**
The 2nd edition of the Duckietown demo was a an internationally coordinated event. Massive Duckietown expositions were built in ETH Zürich and Université de Montréal, and TTI Chicago also joined remotely. The event was broadcast over skype between the locations and thousands of visitors were present between the locations.
The Montréal event was sponsored by Element AI.
**Categories:** Blog, Events
**Tags:** Canada, United States, world
---
### [TV Svizzera - Una "Paperopoli" per la guida autonoma](https://duckietown.com/tv-svizzera-una-paperopoli-per-la-guida-autonoma/)
**Published:** January 26, 2018
**Author:** Andrea Censi
**Content:**
[La TV Svizzera ci intervista](https://www.tvsvizzera.it/tvs/robotica-all-_una--paperopoli--per-la-guida-autonoma/43858086).
**Categories:** Blog, Media Coverage
**Tags:** ETH Zürich, Italy, Switzerland
---
### [A lab for self-driving vehicles](https://duckietown.com/a-lab-for-self-driving-vehicles/)
**Published:** January 30, 2018
**Author:** Andrea Censi
**Content:**
[A lab for self-driving vehicles](https://www.ethz.ch/en/news-and-events/eth-news/news/2018/01/lab-for-self-driving-vehicles.html): profile of the Fall 2017 class.
**Categories:** Blog, Media Coverage
**Tags:** ETH Zürich, world
---
### [Student Day at ST Napoli](https://duckietown.com/student-day-at-st-napoli/)
**Published:** June 5, 2018
**Author:** Duckietown Admin
**Content:**
High school students from around Naples enjoyed Duckietown at STMicroelectronics Research Centre this past May 9th.
\[embedyt\] https://www.youtube.com/watch?v=-TKZ-JZfAzE\[/embedyt\]
**Categories:** Blog, Events, Media Coverage
**Tags:** high school, Italy, world
---
### [Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions](https://duckietown.com/learning-autonomous-systems-an-interdisciplinary-project-based-experience-2/)
**Published:** June 29, 2018
**Author:** Ivano Marocchi
**Content:**
- [ Title: Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions ](https://arxiv.org/abs/1707.07399)
- Authors: Miao Liu, Kavinayan Sivakumar, Shayegan Omidshafiei, Christopher Amato, Jonathan P. How
- Published in IROS 2017.
## Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions
This paper presents a data-driven approach for multi-robot coordination in partially-observable domains based on Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) and macro-actions (MAs). Dec-POMDPs provide a general framework for cooperative sequential decision making under uncertainty and MAs allow temporally extended and asynchronous action execution. To date, most methods assume the underlying Dec-POMDP model is known a priori or a full simulator is available during planning time. Previous methods which aim to address these issues suffer from local optimality and sensitivity to initial conditions. Additionally, few hardware demonstrations involving a large team of heterogeneous robots and with long planning horizons exist. This work addresses these gaps by proposing an iterative sampling based Expectation-Maximization algorithm (iSEM) to learn polices using only trajectory data containing observations, MAs, and rewards. Our experiments show the algorithm is able to achieve better solution quality than the state-of-the-art learning-based methods. We implement two variants of multi-robot Search and Rescue (SAR) domains (with and without obstacles) on hardware to demonstrate the learned policies can effectively control a team of distributed robots to cooperate in a partially observable stochastic environment.
### Did you find this interesting?
Read more Duckietown based papers [here](https://www.duckietown.com/research/papers).
**Categories:** paper, Research
---
### [August 6-8, 2018: Duckietown Summer School, Taiwan](https://duckietown.com/august-6-8-2018-duckietown-summer-school-taiwan/)
**Published:** July 17, 2018
**Author:** Duckietown Admin
**Content:**
The second edition of the [Duckietown Summer School](https://www.duckietown.com/taiwan-summer-school-2018) will take place August 6-8 in Taiwan.
**Categories:** Events
**Tags:** upcoming
---
### [Dec 7, 2018: AI Driving Olympics at NIPS 2018](https://duckietown.com/dec-7-2018-ai-driving-olympics-at-nips-2018/)
**Published:** July 17, 2018
**Author:** Duckietown Admin
**Categories:** Events
**Tags:** upcoming
---
### [May 2019: AI Driving Olympics at ICRA 2019](https://duckietown.com/may-2019-ai-driving-olympics-at-icra-2019/)
**Published:** July 17, 2018
**Author:** Duckietown Admin
**Categories:** Events
**Tags:** upcoming
---
### [Towards blockchain-based robonomics: autonomous agents behavior validation](https://duckietown.com/learning-autonomous-systems-an-interdisciplinary-project-based-experience-3/)
**Published:** July 29, 2018
**Author:** Ivano Marocchi
**Content:**
- [ Title: Towards blockchain-based robonomics: autonomous agents behavior validation ](https://arxiv.org/abs/1805.03241)
- Authors: Konstantin Danilov, Ruslan Rezin, Alexander Kolotov, Ilya Afanasyev
- Published on arXiv on May 8, 2018
## Towards blockchain-based robonomics: autonomous agents behavior validation
The decentralized trading market approach, where both autonomous agents and people can consume and produce services expanding own opportunities to reach goals, looks very promising as a part of the Fourth Industrial revolution. The key component of the approach is a blockchain platform that allows an interaction between agents via liability smart contracts. Reliability of a service provider is usually determined by a reputation model. However, this solution only warns future customers about an extent of trust to the service provider in case it could not execute any previous liabilities correctly. From the other hand a blockchain consensus protocol can additionally include a validation procedure that detects incorrect liability executions in order to suspend payment transactions to questionable service providers. The paper presents the validation methodology of a liability execution for agent-based service providers in a decentralized trading market, using the Model Checking method based on the mathematical model of finite state automata and Temporal Logic properties of interest. To demonstrate this concept, we implemented the methodology in the Duckietown application, moving an autonomous mobile robot to achieve a mission goal with the following behavior validation at the end of a completed scenario.
### Did you find this interesting?
Read more Duckietown based papers [here](https://www.duckietown.com/research/papers).
**Categories:** paper, Research
---
### [Hybrid control and learning with coresets for autonomous vehicles](https://duckietown.com/learning-autonomous-systems-an-interdisciplinary-project-based-experience-4/)
**Published:** July 29, 2018
**Author:** Ivano Marocchi
**Content:**
- [ Title: Hybrid control and learning with coresets for autonomous vehicles ](https://ieeexplore.ieee.org/document/8206612/)
- Guy Rosman, Liam Paull, and Daniela Rus
- Published in IROS 2017
## Hybrid control and learning with coresets for autonomous vehicles
Modern autonomous systems such as driverless vehicles need to safely operate in a wide range of conditions. A potential solution is to employ a hybrid systems approach, where safety is guaranteed in each individual mode within the system. This offsets complexity and responsibility from the individual controllers onto the complexity of determining discrete mode transitions. In this work we propose an efficient framework based on recursive neural networks and coreset data summarization to learn the transitions between an arbitrary number of controller modes that can have arbitrary complexity. Our approach allows us to efficiently gather annotation data from the large-scale datasets that are required to train such hybrid nonlinear systems to be safe under all operating conditions, favoring underexplored parts of the data. We demonstrate the construction of the embedding, and efficient detection of switching points for autonomous and non-autonomous car data. We further show how our approach enables efficient sampling of training data, to further improve either our embedding or the controllers.
### Did you find this interesting?
Read more Duckietown based papers [here](https://www.duckietown.com/research/papers).
**Categories:** paper, Research
---
### [Integration of open source platform Duckietown and gesture recognition as an interactive interface for the museum robotic guide](https://duckietown.com/learning-autonomous-systems-an-interdisciplinary-project-based-experience-5/)
**Published:** July 29, 2018
**Author:** Ivano Marocchi
**Content:**
- [ Title: Integration of open source platform Duckietown and gesture recognition as an interactive interface for the museum robotic guide ](https://ieeexplore.ieee.org/abstract/document/8372718/)
- Authors: Feng-Ching Cheng, Zi-Yu Wang, and Jee-Jee Chen
- Published in 2018 Wireless and Optical Communication Conference
## Integration of open source platform Duckietown and gesture recognition as an interactive interface for the museum robotic guide
In recent years, population aging becomes a serious problem. To decrease the demand for labor when navigating visitors in museums, exhibitions, or libraries, this research designs an automatic museum robotic guide which integrates image and gesture recognition technologies to enhance the guided tour quality of visitors. The robot is a self-propelled vehicle developed by ROS (Robot Operating System), in which we achieve the automatic driving based on the function of lane-following via image recognition. This enables the robot to lead guests to visit artworks following the preplanned route. In conjunction with the vocal service about each artwork, the robot can convey the detailed description of the artwork to the guest. We also design a simple wearable device to perform gesture recognition. As a human machine interface, the guest is allowed to interact with the robot by his or her hand gestures. To improve the accuracy of gesture recognition, we design a two phase hybrid machine learning-based framework. In the first phase (or training phase), k-means algorithm is used to train historical data and filter outlier samples to prevent future interference in the recognition phase. Then, in the second phase (or recognition phase), we apply KNN (k-nearest neighboring) algorithm to recognize the hand gesture of users in real time. Experiments show that our method can work in real time and get better accuracy than other methods.
### Did you find this interesting?
Read more Duckietown based papers [here](https://www.duckietown.com/research/papers).
**Categories:** paper, Research
---
### [Deep Trail-Following Robotic Guide Dog in Pedestrian Environments for People who are Blind and Visually Impaired - Learning from Virtual and Real Worlds](https://duckietown.com/learning-autonomous-systems-an-interdisciplinary-project-based-experience-7/)
**Published:** July 29, 2018
**Author:** Ivano Marocchi
**Content:**
- [ Title: Deep Trail-Following Robotic Guide Dog in Pedestrian Environments for People who are Blind and Visually Impaired - Learning from Virtual and Real Worlds ](https://www.cs.umb.edu/~craigyu/papers/guidedog.pdf)
- Authors: Tzu-Kuan Chuang, Ni-Ching Lin, Jih-Shi Chen, Chen-Hao Hung, Yi-Wei Huang, Chunchih Teng, Haikun Huang, Lap-Fai Yu, Laura Giarre, and Hsueh-Cheng Wang
- Published in ICRA 2018
## Deep Trail-Following Robotic Guide Dog in Pedestrian Environments for People who are Blind and Visually Impaired - Learning from Virtual and Real Worlds
Navigation in pedestrian environments is critical to enabling independent mobility for the blind and visually impaired (BVI) in their daily lives. White canes have been commonly used to obtain contact feedback for following walls, curbs, or man-made trails, whereas guide dogs can assist in avoiding physical contact with obstacles or other pedestrians. However, the infrastructures of tactile trails or guide dogs are expensive to maintain. Inspired by the autonomous lane following of self-driving cars, we wished to combine the capabilities of existing navigation solutions for BVI users. We proposed an autonomous, trail-following robotic guide dog that would be robust to variances of background textures, illuminations, and interclass trail variations. A deep convolutional neural network (CNN) is trained from both the virtual and realworld environments. Our work included major contributions: 1) conducting experiments to verify that the performance of our models trained in virtual worlds was comparable to that of models trained in the real world; 2) conducting user studies with 10 blind users to verify that the proposed robotic guide dog could effectively assist them in reliably following man-made trails.
### Did you find this interesting?
Read more Duckietown based papers [here](https://www.duckietown.com/research/papers).
**Categories:** paper, Research
---
### [Learning autonomous systems — An interdisciplinary project-based experience](https://duckietown.com/learning-autonomous-systems-an-interdisciplinary-project-based-experience/)
**Published:** July 29, 2018
**Author:** Liam Paull
**Content:**
- [ Title: Learning autonomous systems — An interdisciplinary project-based experience ](https://ieeexplore.ieee.org/abstract/document/8190555)
- Authors: Brian Page, Saeedeh Ziaeefard, Barzin Moridian, Nina Mahmoudian
- Published: 2017 IEEE Frontiers in Education Conference
## Learning autonomous systems — An interdisciplinary project-based experience
With the increased influence of automation into every part of our lives, tomorrow’s engineers must be capable working with autonomous systems. The explosion of automation and robotics has created a need for a massive increase in engineers who possess the skills necessary to work with twenty-first century systems. Autonomous Systems (MEEM4707) is a new senior/graduate level elective course with goals of: 1) preparing the next generation of skilled engineers, 2) creating new opportunities for learning and well informed career choices, 3) increasing confidence in career options upon graduation, and 4) connecting academic research to the students world. Presented in this paper is the developed curricula, key concepts of the project-based approach, and resources for other educators to implement a similar course at their institution. In the course, we cover the fundamentals of autonomous robots in a hands-on manner through the use of a low-cost mobile robot. Each student builds and programs their own robot, culminating in operation of their autonomous mobile robot in a miniature city environment. The concepts covered in the course are scalable from middle school through graduate school. Evaluation of student learning is completed using pre/post surveys, student progress in the laboratory environment, and conceptual examinations.
### Did you find this interesting?
Read more Duckietown based papers [here](https://www.duckietown.com/research/papers).
**Categories:** paper, Research
---
### [Duckietown: An open, inexpensive and flexible platform for autonomy education and research](https://duckietown.com/duckietown-an-open-inexpensive-and-flexible-platform-for-autonomy-education-and-research/)
**Published:** July 29, 2018
**Author:** Liam Paull
**Content:**
- [ Title: Duckietown: An open, inexpensive and flexible platform for autonomy education and research ](https://ieeexplore.ieee.org/document/7989179)
- Authors: Liam Paull; Jacopo Tani; Heejin Ahn; Javier Alonso-Mora; Luca Carlone; Michal Cap; Yu Fan Chen; Changhyun Choi; Jeff Dusek; Yajun Fang; Daniel Hoehener; Shih-Yuan Liu; Michael Novitzky; Igor Franzoni Okuyama; Jason Pazis; Guy Rosman; Valerio Varricchio; Hsueh-Cheng Wang; Dmitry Yershov; Hang Zhao; Michael Benjamin; Christopher Carr; Maria Zuber; Sertac Karaman; Emilio Frazzoli; Domitilla Del Vecchio; Daniela Rus; Jonathan How; John Leonard; Andrea Censi
- Published: 2017 IEEE International Conference on Robotics and Automation (ICRA)
## Duckietown: An open, inexpensive and flexible platform for autonomy education and research
Duckietown is an open, inexpensive and flexible platform for autonomy education and research. The platform comprises small autonomous vehicles (“Duckiebots”) built from off-the-shelf components, and cities (“Duckietowns”) complete with roads, signage, traffic lights, obstacles, and citizens (duckies) in need of transportation. The Duckietown platform offers a wide range of functionalities at a low cost. Duckiebots sense the world with only one monocular camera and perform all processing onboard with a Raspberry Pi 2, yet are able to: follow lanes while avoiding obstacles, pedestrians (duckies) and other Duckiebots, localize within a global map, navigate a city, and coordinate with other Duckiebots to avoid collisions. Duckietown is a useful tool since educators and researchers can save money and time by not having to develop all of the necessary supporting infrastructure and capabilities. All materials are available as open source, and the hope is that others in the community will adopt the platform for education and research.
### Did you find this interesting?
Read more Duckietown based papers [here](https://www.duckietown.com/research/papers).
**Categories:** education, outreach, paper, Research
---
### [Swiss TV interviews ETH Zurich instructors](https://duckietown.com/swiss-tv-intervies-eth-zurich-instructors/)
**Published:** July 30, 2018
**Author:** Andrea Censi
**Content:**
[An Italian interview](https://www.tvsvizzera.it/tvs/robotica-all-_una--paperopoli--per-la-guida-autonoma/43858086) to the Zürich students and instructors in the occasion of Demo Day 2017.
**Categories:** Blog, Media Coverage
**Tags:** ETH Zürich, Italy, Switzerland
---
### [Die AI-Fahrolympiade auf der NIPS 2018](https://duckietown.com/die-ai-fahrolympiade-auf-der-nips-2018/)
**Published:** August 7, 2018
**Author:** Liam Paull
**Content:**
Autoren:
Andrea Censi Liam Paull, Jacopo Tani, Julian Zilly, Thomas Ackermann, Oscar Beijbom, Berabi Berkai, Gianmarco Bernasconi, Anne Kirsten Bowser, Simon Bing, Pin-Wei David Chen, Yu-Chen Chen, Maxime Chevalier-Boisvert, Breandan Considine, Andrea Daniele, Justin De Castri, Maurilio Di Cicco, Manfred Diaz, Paul Aurel Diederichs, Florian Golemo, Ruslan Hristov, Lily Hsu, Yi-Wei Daniel Huang, Chen-Hao Peter Hung, Qing-Shan Jia, Julien Kindle, Dzenan Lapandic, Cheng-Lung Lu, Sunil Mallya, Bhairav Mehta, Aurel Neff, Eryk Nice, Yang-Hung Allen Ou, Abdelhakim Qbaich, Josefine Quack, Claudio Ruch, Adam Sigal, Niklas Stolz, Alejandro Unghia, Ben Weber, Sean Wilson, Zi-Xiang Xia, Timothius Victorio Yasin, Nivethan Yogarajah, Yoshua Bengio, Tao Zhang, Hsueh-Cheng Wang, Matthew Walter, Stefano Soatto, Magnus Egerstedt, Emilio Frazzoli,
Veröffentlicht auf dem RSS-Workshop über neue Benchmarks, Metriken und Wettbewerbe für Robotisches Lernen.
Link: [Verfügbar hier](https://www.duckietown.com/wp-content/uploads/2018/08/nips-2018-live-4-min.pdf)
**Categories:** Blog, Manual
---
### [Las Olimpiadas AI Driving en NIPS 2018](https://duckietown.com/las-olimpiadas-ai-driving-en-nips-2018/)
**Published:** August 9, 2018
**Author:** Liam Paull
**Content:**
Autores:
Andrea Censi Liam Paull, Jacopo Tani, Julian Zilly, Thomas Ackermann, Oscar Beijbom, Berabi Berkai, Gianmarco Bernasconi, Anne Kirsten Bowser, Simon Bing, Pin-Wei David Chen, Yu-Chen Chen, Maxime Chevalier-Boisvert, Breandan Considine, Andrea Daniele, Justin De Castri, Maurilio Di Cicco, Manfred Diaz, Paul Aurel Diederichs, Florian Golemo, Ruslan Hristov, Lily Hsu, Yi-Wei Daniel Huang, Chen-Hao Peter Hung, Qing-Shan Jia, Julien Kindle, Dzenan Lapandic, Cheng-Lung Lu, Sunil Mallya, Bhairav Mehta, Aurel Neff, Eryk Nice, Yang-Hung Allen Ou, Abdelhakim Qbaich, Josefine Quack, Claudio Ruch, Adam Sigal, Niklas Stolz, Alejandro Unghia, Ben Weber, Sean Wilson, Zi-Xiang Xia, Timothius Victorio Yasin, Nivethan Yogarajah, Yoshua Bengio, Tao Zhang, Hsueh-Cheng Wang, Matthew Walter, Stefano Soatto, Magnus Egerstedt, Emilio Frazzoli,
Publicado en *RSS Workshop on New Benchmarks, Metrics, and Competitions for Robotic Learning*
Link: [Disponible aquí](https://www.duckietown.com/wp-content/uploads/2018/08/nips-2018-live-4-min.pdf)
**Categories:** Blog, Manual
---
### [Driving autonomy education at BFH with Prof. Affolter](https://duckietown.com/driving-autonomy-education-with-peter-affolter/)
**Published:** November 24, 2023
**Author:** Duckietown Admin
**Content:**
Bern, Switzerland: Prof. Peter Affolter, Head of the automotive department at Berner Fachhochschule (BFH), shares his experience creating a curriculum in autonomy education with Duckietown. Quick links Bern University of Applied Science Prof. Peter Affolter @ BFH Prof. Peter Affolter Driving autonomy
**Bern, Switzerland:** Prof. Peter Affolter, Head of the automotive department at Berner Fachhochschule (BFH), shares his experience creating a curriculum in autonomy education with Duckietown.
##### Quick links
- [ Bern University of Applied Science ](https://www.bfh.ch/en/)
- [ Prof. Peter Affolter @ BFH ](https://www.bfh.ch/de/ueber-die-bfh/personen/kfkwyfuljpjv/)
- [ Prof. Peter Affolter ](https://www.linkedin.com/in/peter-affolter-653b8b9b/?originalSubdomain=ch)
# Driving autonomy education with Duckietown
## Prof. Affolter at the Bern University of Applied Sciences introduces autonomy education in the automotive curriculum
##### Hi! Thank you for taking the time to chat with us. I'd like to start by asking you to introduce yourself, and tell us who you are and what you do.
Certainly! My name is Peter Affolter. I am at the head of the automotive department at the Bern University of Applied Science. I work as a lecturer in autonomous driving, vehicle communications, and related areas.
##### How did you get to know Duckietown?
The first time I saw it was probably two or three years ago. I came across it by chance while Googling on the internet. I was looking for information about lessons and classes on autonomous driving, a very complicated topic. I was searching for an easy platform, especially hardware, to simulate a simple vehicle. Then, I stumbled upon Duckietown. Initially, it looked a bit childish, but I was surprised by its concept, quality, and comprehensive documentation, including lessons slides for this complex theory. I decided to focus on Duckietown, especially on the massive online open course hosted on edX \[[Self-Driving Cars with Duckietown](https://duckietown.com/mooc/ "Self-Driving Cars with Duckietown")\]. I also really enjoyed the great support from all the stakeholders and universities involved in this initiative.
##### Did you find the "Self-Driving Cars with Duckietown", the massive open online course (MOOC) on edX, helpful?
Yes, at that time, there was only the archive available on edX.org, and I registered there. I couldn’t participate in real-time, but I went through the materials myself. I also contacted the Duckietown staff, and they provided me with support and missing documents. I adapted the content for my own class, focusing on bachelor-level students, parallel to the massive online open course from edX.
##### Very interesting, so you adapted the materials for your course. How did you set up the class, did you find any challenges?
It was clear to me that I could introduce our students, future automotive engineers, to autonomous driving. I wanted to give them an overview of the potential, limits, obstacles, and areas needing progress. Since they were not programmers or IT technologists, I needed to simplify it. I decided to conduct whole-class courses using video material from the MOOC. I adapted the quizzes and provided them with virtual machines, avoiding Linux and network issues. They could use the vehicles hosted on the BFH server farm through a browser terminal, even utilizing the Duckiebots. It became an infrastructure for exercises from simulation to reality, based on Duckietown classes.
##### How did the teaching experience go?
The first class had potential for improvement, but we adopted a flipped classroom approach with four lessons a week for 16 weeks. Students studied theory at home, and class time was for exercises, discussions, and additional explanations. The students enjoyed it, and while they didn’t grasp every detail, they felt proud and motivated. There’s still plenty of room for improvement in the next classes. They grasped the feeling and had the chance to work on neural networks, image manipulations, and more. I made good progress, and there’s still plenty of material to explore and implement in future classes.
> Honestly, I haven't found another hardware platform as good as Duckietown for my needs. It's a simple platform with a fun approach, well-documented, and with a really reactive community. Even compared to other commercial products, Duckieown stands out. It fulfills all the needs from beginners to experts.
>
> Prof. Peter Affolter, Head of Automotive Dept. at BFH
##### What were your overall impressions introducing autonomy education to your students with Duckietown?
Honestly, I haven’t found another hardware platform as good as Duckietown for my needs. It’s a simple platform with a fun approach, well-documented, and with a really reactive community. Even compared to other commercial products, Duckietown stands out. It fulfills all the needs from beginners to experts.
**Thank you again for sharing your experience with us. Best of luck in your next classes!**
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences. It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
**Categories:** Blog, News, People
---
### [Problems with local evaluator (dts challenges evaluate)](https://duckietown.com/problems-with-local-evaluator-dts-challenges-evaluate/)
**Published:** October 20, 2018
**Author:** Andrea Censi
**Content:**
We have reports of problems with the `dts challenges evaluate` command (local evaluator).
This is a tricky command because it spawns a Docker container that spawns other Docker containers etc.
A workaround is described at [here](http://docs.duckietown.com/DT18/AIDO/out/cm_first.html#sub:cm-local) and
it involves running the native Python code for the evaluator.
**Categories:** Uncategorized
**Tags:** AI Driving Olympics
---
### [Interactive Learning with Corrective Feedback for Policies based on Deep Neural Networks](https://duckietown.com/interactive-learning-with-corrective-feedback-for-policies-based-on-deep-neural-networks/)
**Published:** October 23, 2018
**Author:** Jacopo Tani
**Content:**
- [ Title: Interactive Learning with Corrective Feedback for Policies based on Deep Neural Networks ](https://arxiv.org/pdf/1810.00466.pdf)
- Authors: Rodrigo Pérez-Dattari, Carlos Celemin, Javier Ruiz-del-Solar, Jens Kober
- Published: International Symposium on Experimental Robotics (ISER 2018)
## Interactive Learning with Corrective Feedback for Policies based on Deep Neural Networks
Deep Reinforcement Learning (DRL) has become a powerful strategy to
solve complex decision making problems based on Deep Neural Networks (DNNs).
However, it is highly data demanding, so unfeasible in physical systems for most
applications. In this work, we approach an alternative Interactive Machine Learning (IML) strategy for training DNN policies based on human corrective feedback,
with a method called Deep COACH (D-COACH). This approach not only takes advantage of the knowledge and insights of human teachers as well as the power of
DNNs, but also has no need of a reward function (which sometimes implies the
need of external perception for computing rewards). We combine Deep Learning
with the COrrective Advice Communicated by Humans (COACH) framework, in
which non-expert humans shape policies by correcting the agent’s actions during
execution. The D-COACH framework has the potential to solve complex problems
without much data or time required.
Experimental results validated the efficiency of the framework in three different problems (two simulated, one with a real robot),with state spaces of low and high dimensions, showing the capacity to successfully learn policies for continuous action spaces like in the Car Racing and Cart-Pole problems faster than with DRL.
### Introduction
Deep Reinforcement Learning (DRL) has obtained unprecedented results in decisionmaking problems, such as playing Atari games \[1\], or beating the world champion inGO \[2\].
Nevertheless, in robotic problems, DRL is still limited in applications with
real-world systems \[3\]. Most of the tasks that have been successfully addressed with
DRL have two common characteristics: 1) they have well-specified reward functions, and 2) they require large amounts of trials, which means long training periods
(or powerful computers) to obtain a satisfying behavior. These two characteristics
can be problematic in cases where 1) the goals of the tasks are poorly defined or
hard to specify/model (reward function does not exist), 2) the execution of many
trials is not feasible (real systems case) and/or not much computational power or
time is available, and 3) sometimes additional external perception is necessary for
computing the reward/cost function.
On the other hand, Machine Learning methods that rely on transfer of human
knowledge, Interactive Machine Learning (IML) methods, have shown to be time efficient for obtaining good performance policies and may not require a well-specified
reward function; moreover, some methods do not need expert human teachers for
training high performance agents \[4–6\]. In previous years, IML techniques were
limited to work with low-dimensional state spaces problems and to the use of function approximation such as linear models of basis functions (choosing a right basis
function set was crucial for successful learning), in the same way as RL. But, as
DRL have showed, by approximating policies with Deep Neural Networks (DNNs)
it is possible to solve problems with high-dimensional state spaces, without the need
of feature engineering for preprocessing the states. If the same approach is used in
IML, the DRL shortcomings mentioned before can be addressed with the support of
human users who participate in the learning process of the agent.
This work proposes to extend the use of human corrective feedback during task
execution to learn policies with state spaces of low and high dimensionality in continuous action problems (which is the case for most of the problems in robotics)
using deep neural networks.
We combine Deep Learning (DL) with the corrective advice based learning
framework called COrrective Advice Communicated by Humans (COACH) \[6\],
thus creating the Deep COACH (D-COACH) framework. In this approach, no reward functions are needed and the amount of learning episodes is significantly reduced in comparison to alternative approaches. D-COACH is validated in three different tasks, two in simulations and one in the real-world.
### Conclusions
This work presented D-COACH, an algorithm for training policies modeled with
DNNs interactively with corrective advice. The method was validated in a problem
of low-dimensionality, along with problems of high-dimensional state spaces like
raw pixel observations, with a simulated and a real robot environment, and also
using both simulated and real human teachers.
The use of the experience replay buffer (which has been well tested for DRL) was
re-validated for this different kind of learning approach, since this is a feature not
included in the original COACH. The comparisons showed that the use of memory
resulted in an important boost in the learning speed of the agents, which were able
to converge with less feedback, and to perform better even in cases with a significant
amount of erroneous signals.
The results of the experiments show that teachers advising corrections can train
policies in fewer time steps than a DRL method like DDPG. So it was possible
to train real robot tasks based on human corrections during the task execution, in
an environment with a raw pixel level state space. The comparison of D-COACH
with respect to DDPG, shows how this interactive method makes it more feasible
to learn policies represented with DNNs, within the constraints of physical systems.
DDPG needs to accumulate millions of time steps of experience in order to obtain
### Did you find this interesting?
Read more Duckietown based papers [here](https://www.duckietown.com/about/papers).
**Categories:** paper, Research
**Tags:** chile, research, science
---
### [Updated Duckietown-challenges server fixes speed problems; updated "dts commands evaluate"](https://duckietown.com/updated-duckietown-challenges-server-fixes-speed-problems-updated-dts-commands-evaluate/)
**Published:** October 24, 2018
**Author:** Andrea Censi
**Content:**
As we have more participants, the [Duckietown Challenges Server](https://challenges.duckietown.com)
started to feel slow. The reason: we were a bit lazy and some pages had O(n) implementations where O(1) was needed – loading all challenges/submissions/etc.
We also updated the `"dts challenges evaluate"` command to be more robust. Please continue to report bugs
as this part is fragile by nature — running containers that spawn other containers on the user’s machines.
**Categories:** Uncategorized
**Tags:** AI Driving Olympics
---
### [Server capacity increased; leaderboards logic updated.](https://duckietown.com/server-capacity-increased-leaderboards-logic-updated/)
**Published:** October 31, 2018
**Author:** Andrea Censi
**Content:**
We increased the server capacity. This should mitigate the slowness issues due to the unexpected number of participants/submissions.
We updated the logic of how we compute the leaderboard; now there can be only one submission per user.
**Categories:** Uncategorized
**Tags:** AI Driving Olympics
---
### [Update to challenge LF, LFV evaluation code](https://duckietown.com/update-to-challenge-lf-lfv-evaluation-code/)
**Published:** November 7, 2018
**Author:** Andrea Censi
**Content:**
We are going to roll out an improvement to the LF and LFV challenges competitions. This change fixes the following problems:
- The robot will always start in the right lane - a legal position.
- The evaluation and visualization code are going to be richer, with more statistics plotted ([example](http://duckietown-ai-driving-olympics-1.s3.amazonaws.com/v3/testing/by-value/sha256/b4e5a76437a318544a2a5f210de9c594d6fdabaa236fe0dd0dc311e99f6979c1)).
- The evaluation rulebook is slightly changed to address a couple of bugs of how the metrics were computed.
What is going to happen is the following: - The moment that we update the evaluation code, all existing submissions are set back to the state of "evaluation".
- The evaluators will then re-evaluate all of them. This will take 2-3 hours.
During this time the leaderboards are going to be blank, and slowly will re-populate as the evaluators do their job. (To speed up evaluation of your submissions, you can run dts challenges evaluator.)
**Categories:** Uncategorized
**Tags:** AI Driving Olympics
---
### [AI-DO1 Submission Deadline: Thursday Dec 6 at 11:59pm PST](https://duckietown.com/ai-do1-submission-deadline-thursday-dec-6-at-1159pm-pst/)
**Published:** December 4, 2018
**Author:** Jacopo Tani
**Content:**
We’re just about at the end of the road for the 2018 AI Driving Olympics.
There’s certainly been some action on the leaderboard these last few days and it’s going down to the wire. Don’t miss your chance to see you name up there and win the amazing prizes donated by nuTonomy and Amazon AWS!
Submissions will close at **11:59pm PST on Thursday Dec. 6**.
Please join us at NeurIPS for the live competition 3:30-5:00pm EST in room 511!
**Categories:** Events, News, Uncategorized
**Tags:** AI Driving Olympics, AI-DO, news, NIPS
---
### [AI-DO 1 at NeurIPS report. Congratulations to our winners!](https://duckietown.com/ai-do-at-neurips-is-over-congratulations-to-our-winners/)
**Published:** December 11, 2018
**Author:** Jacopo Tani
**Content:**
## The winners of AIDO-1 at NeurIPS
There was a great turnout for the first AI Driving Olympics competition, which took place at the NeurIPS conference in Montreal, Canada on Dec 8, 2018. In the finals, the submissions from the top five competitors were run from five different locations on the competition track.
Our top five competitors were awarded $3000 worth of AWS Credits (thank you AWS!) and a trip to one of nuTonomy’s offices for a ride in one of their self-driving cars (thanks APTIV!)
## WINNER
Team Panasonic R&D Center Singapore & NUS
(Wei Gao)
Check out the [submission](https://challenges.duckietown.com/v3/humans/submissions/1633).
*The approach:* We used the random template for its flexibility and created a debug framework to test the algorithm. After that, we created one python package for our algorithm and used the random template to directly call it. The algorithm basically contains three parts: 1. Perception, 2. Prediction and 3. Control. Prediction plays the most important role when the robot is at the sharp turn where the camera can not observe useful information.
## 2nd Place
Jon Plante
Check out the [submission](https://challenges.duckietown.com/v3/humans/submissions/1684).
*The approach*: “I tried and imitate what a human does when he follows a lane. I believe the human tries to center itself at all times in the lane using the two lines as guides. I think the human implicitly projects the two lines into the horizon and where they intersect is where the human directs the vehicle towards.”
## 3rd Place
Vincent Mai
Check out the [submission](https://challenges.duckietown.com/v3/humans/submissions/1033).
*The approach*: “The AI-DO application I made was using the ROS lane following baseline. After running it out of the box, I noticed a couple of problems and corrected them by changing several parameters in the code.”
## 4th Place
Team JetBrains
(Mikita Sazanovich)
Check out the [submission](https://challenges.duckietown.com/v3/humans/submissions/1859).
*The approach:* “We used [our framework](https://github.com/iasawseen/MultiServerRL) for parallel deep reinforcement learning. Our network consisted of five convolutional layers (1st layer with 32 9×9 filters, each following layer with 32 5×5 filters), followed by two fully connected layers (with 768 and 48 neurons) that took as an input four last frames downsampled to 120 by 160 pixels and filtered for white and yellow color. We trained it with Deep Deterministic Policy Gradient algorithm (Lillicrap et al. 2015). The training was done in three stages: first, on a full track, then on the most problematic regions, and then on a full track again.”
## 5th Place
Team SAIC Moscow
(Anton Mashikhin)
Check out the [submission](https://challenges.duckietown.com/v3/humans/submissions/1579).
*The approach:* Our solution is based on reinforcement learning algorithm. We used a Twin delayed DDPG and ape-x like distributed scheme. One of the key insights was to add PID controller as an additional explorative policy. It has significantly improved learning speed and quality
## A few photos from the day
[](https://duckietown.com/wp-content/uploads/2018/12/Jacopo-Tani-IMG_5133.jpg) Duckies invaded the conference venue. They could be heard throughout the halls all day. [](https://duckietown.com/wp-content/uploads/2018/12/Jacopo-Tani-IMG_5054.jpg) Getting our Duckiebots charged [](https://duckietown.com/wp-content/uploads/2018/12/Jacopo-Tani-IMG_5059.jpg) A little event advertising
[](https://duckietown.com/wp-content/uploads/2018/12/Jacopo-Tani-IMG_5100.jpg) Test run [](https://duckietown.com/wp-content/uploads/2018/12/Jacopo-Tani-IMG_5036.jpg) Competition track under construction [](https://duckietown.com/wp-content/uploads/2018/12/Jacopo-Tani-IMG_5255.jpg) The competition track
[](https://duckietown.com/wp-content/uploads/2018/12/Kirsten-Bowser-IMG_20181208_171949.jpg) The demonstration track [](https://duckietown.com/wp-content/uploads/2018/12/Maxime-Chevalier-Boisvert-IMG-4882.jpg) The competition is underway! [](https://duckietown.com/wp-content/uploads/2018/12/Jacopo-Tani-IMG_5273.jpg) We all “quacked” instead of clapped after each run
**Categories:** Events, News
**Tags:** AI Driving Olympics, announcement, competition, event, montreal, neurips, news, NIPS
---
### [Countdown to AI-DO 2!](https://duckietown.com/countdown-to-ai-do-2/)
**Published:** February 19, 2019
**Author:** Jacopo Tani
**Content:**
Didn’t get a chance to compete in the AI Driving Olympics at NeurIPS this past December? Not to worry! The second iteration of the AI-DO will take place at [ICRA](https://www.icra2019.org/) this May. Get your engines and algorithms up and running by checking out the information on the AI-DO [website](/?page_id=176).
**Categories:** dep-News, News
**Tags:** AI Driving Olympics, AI-DO, competition, event, icra, news
---
### [Press Release AI-DO2](https://duckietown.com/press-release-ai-do2/)
**Published:** April 15, 2019
**Author:** Jacopo Tani
**Content:**
\[pdfviewer width=”600px” height=”849px” beta=”false”\]https://www.duckietown.com/wp-content/uploads/2019/04/AIDO2-Press-release-EN.pdf\[/pdfviewer\]
**Categories:** News, Uncategorized
---
### [AI-DO technical updates](https://duckietown.com/ai-do-technical-updates/)
**Published:** April 24, 2019
**Author:** Andrea Censi
**Content:**
Here are some technical updates regarding the competition. Thanks for all the bug reports via Github and Slack!
### Changes to platform model in simulations
We have changed the purely kinematic model in the simulations with one that is more similar to the real robots obtained by system identification. You can find the model [here](https://github.com/duckietown/duckietown-world/blob/aido2/src/duckietown_world/world_duckietown/pwm_dynamics.py). Properties: - The inputs to the model are the two PWM signals to the wheels, left and right. (**not** \[speed, omega\] like last year)
- The maximum velocity is ~2 m/s. The rise time is about 1 second.
- There is a simulated delay of 100 ms.
We will slightly perturb the parameters of the model in the future to account for robot-robot variations, but this is not implemented yet. All the submissions have been re-evaluated. You can see the difference between the two models *purely kinematic platform model* *more realistic platform model* The new model is much more smooth. Overall we expect that the new model makes the competition easier both in simulation, and obviously, in the transfer.
### Infrastructure changes
- We have update the Duckietown Shell and commands several times to fix a few reported bugs.
- We have started with provisioning AWS cloud evaluators. There are still sporadic problems. You should know that if your job fails with the `host-error` code, the system thinks it is a problem of the evaluator and it will try on another evaluator.
### Open issues
- Some timeouts are a bit tight. Currently we allow 20 minutes like for NeurIPS, but this year we have much more realistic simulation and better visualization code that take more time. If your submission fails after 20 minutes of evaluation, this is the reason.
- We are still working on the glue code for running the submissions on the real robots. Should be a couple of days away.
- Some of the changes to the models/protocol above are not in the docs yet.
**Categories:** dep-News
**Tags:** AI Driving Olympics, AI-DO
---
### [AI-DO 2 Validation and Testing Registration](https://duckietown.com/ai-do-2-validation-and-testing-registration/)
**Published:** May 2, 2019
**Author:** Jacopo Tani
**Content:**
We are in the final countdown to AI-DO 2 at ICRA!
Now is the time to let us know if you will be using the validation and testing facilities at the Duckietown competition ground. Please register below!
\[ninja\_form id=27\]
**Categories:** dep-News, Events, News
**Tags:** AI Driving Olympics, AI-DO, event, icra, news, register
---
### [Update to Dynamics Model in Duckietown Simulator](https://duckietown.com/update-to-dynamics-model-in-duckietown-simulator/)
**Published:** May 13, 2019
**Author:** Liam Paull
**Content:**
We have implemented an improved dynamics model in the simulator. If you are using the simulator to:
- Train your agent with reinforcement learning
- Generate data for imitation learning
- Test and debug your submission
then you may want to retrain/retest with the new dynamics model. This model is much closer to the true Duckiebot and should permit much easier transfer from simulation to the real robot hardware.
**Categories:** Uncategorized
**Tags:** AI Driving Olympics, AI-DO
---
### [AI-DO Robotarium Evaluations Underway](https://duckietown.com/ai-do-robotarium-evaluations-underway/)
**Published:** May 16, 2019
**Author:** Liam Paull
**Content:**
# Autolab evaluations underway
We have started evaluating the submissions in our Duckietown “Robotarium” (aka Autolab):
*Duckiebot onboard camera feed*
*Robotarium watchtower camera feed*
**To queue your submissions for robotarium evaluation, please follow these instructions:**
You need to use the –challenge option to specify 3 challenges: the two simulated ones (testing and validation) and the hardware one:
- dts challenges submit –challenge aido2-LF-sim-validation,aido2-LF-sim-testing,aido2-LF-real-validation
- dts challenges submit –challenge aido2-LFV-sim-validation,aido2-LFV-sim-testing,aido2-LFV-real-validation
- dts challenges submit –challenge aido2-LFV-sim-validation,aido2-LFVI-sim-testing,aido2-LFVI-real-validation
We will evaluate submissions by participants that are in the top part of the leaderboard in the simulated testing challenge.
The robotarium evaluations are limited, and we will do them in a round robin strategy for each user. We aim to evaluate all in the top 10 of the simulated challenge; and then more if there is the possibility.
Participants can have multiple submissions in the “real” challenges. We will evaluate first according to “user priority” or by most recent. The priority is settable through the web interface by using the top right button.
### Deadlines
The challenges will close May 21 at 8pm Montreal (EDT) time. Please check the server timestamp for the precise time in your time zone.
**Categories:** Blog, Events, Research
**Tags:** AI Driving Olympics, AI-DO, announcement
---
### [Congratulations to the winners of the second edition of the AI Driving Olympics!](https://duckietown.com/congratulations-to-the-winners-of-the-second-edition-of-the-ai-driving-olympics/)
**Published:** June 3, 2019
**Author:** Jacopo Tani
**Content:**
#### Team JetBrains came out on top on all 3 challenges
It was a busy (and squeaky) few days at the International Conference on Robotics and Automation in Montreal for the organizers and competitors of the AI Driving Olympics.
The finals were kicked off by a semifinals round, where we the top 5 submissions from the Lane Following in Simulation leaderboard. The finalists ([JBRRussia](https://challenges.duckietown.com/v4/humans/users/1590) and [MYF](https://challenges.duckietown.com/v4/humans/users/1360)) moved forward to the more complicated challenges of Lane Following with Vehicles and Lane Following with Vehicles and Intersections.
 Results from the AI-DO2 Finals event on May 22, 2019 at ICRA
If you couldn’t make it to the event and missed the live stream on Facebook, here’s a short video of the first run of the JetBrains Lane Following submission.
Thanks to everyone that competed, dropped in to say hello, and cheered on the finalists by sending the song of the Duckie down the corridors of the Palais des Congrès.
### A few pictures from the event
[](https://duckietown.com/wp-content/uploads/2019/05/20190520_123714-e1559314336236.jpg) the validation tracks [](https://duckietown.com/wp-content/uploads/2019/05/20190522_115501.jpg) Team meeting [](https://duckietown.com/wp-content/uploads/2019/05/20190521_132942.jpg) Competitors at work
[](https://duckietown.com/wp-content/uploads/2019/05/20190520_155612.jpg) DuckiePond [](https://duckietown.com/wp-content/uploads/2019/05/20190520_152655.jpg) Duckietown Junior [](https://duckietown.com/wp-content/uploads/2019/05/20190521_141547.jpg) A young Forbes roboticist
[](https://duckietown.com/wp-content/uploads/2019/05/20190521_140101.jpg) Street name matching game! [](https://duckietown.com/wp-content/uploads/2019/05/IMG_20190520_144506.jpg) Street name matching game! [](https://duckietown.com/wp-content/uploads/2019/05/IMG_20190521_161042.jpg) Street name matching game!
[](https://duckietown.com/wp-content/uploads/2019/05/20190521_153514-e1559325436132.jpg) Street name matching game! [](https://duckietown.com/wp-content/uploads/2019/05/james-in-duckietown.jpg) Street name matching game! [](https://duckietown.com/wp-content/uploads/2019/05/20190521_150720-e1559320315461.jpg) Street name matching game!
[](https://duckietown.com/wp-content/uploads/2019/05/20190522_142445.jpg) Countdown to the finals [](https://duckietown.com/wp-content/uploads/2019/05/20190522_130427.jpg) Last minute touches on the slides [](https://duckietown.com/wp-content/uploads/2019/05/20190522_142547.jpg) Platform gurus
[](https://duckietown.com/wp-content/uploads/2019/05/20190522_151541.jpg) Intro to the AI-DO, it’s as easy as a few lines of code [](https://duckietown.com/wp-content/uploads/2019/05/20190522_144527.jpg) The audience of AIDO2 [](https://duckietown.com/wp-content/uploads/2019/05/20190522_154540.jpg) JetBrains saying a word or two
[](https://duckietown.com/wp-content/uploads/2019/05/aido2group-winners.001.jpeg) Group shot with our winners!
## #### Don't know much about the AI Driving Olympics?
It is an accessible and reproducible autonomous car competition designed with straightforward standardized hardware, software and interfaces.
#### Get Started
**Step 1: Build and test your agent with our available templates and baselines**
**Step 2: Submit to a challenge**
[https://challenges.duckietown.com](https://challenges.duckietown.com/v4/)
Check out the leaderboard
View your submission in simulation
**Step 3: Run your submission on a robot**
in a Robotarium
**Categories:** Blog, dep-News, Events, News
**Tags:** AI Driving Olympics, AI-DO, announcement, competition, global, icra, news, winner
---
### [Duckietown Workshop at RoboCup Junior](https://duckietown.com/duckietown-workshop-at-robocup-junior/)
**Published:** June 19, 2019
**Author:** Jacopo Tani
**Content:**
## Duckietown Workshop at RoboCup Junior 2019
In collaboration with the RoboCup Federation, the Duckietown Foundation will be offering workshops at RoboCup 2019 in Sydney, Australia, providing a hands-on introduction to the Duckietown platform.
We will be hosting three one-day workshops as part of RoboCup 2019 from July 4-6, 2019 for teachers, students, and independent learners who are interested in finding out more about the Duckietown platform. Attendance is completely free and everyone is welcome to apply, even if you are not participating in RoboCup.
There are no formal requirements, though basic familiarity with GNU/Linux and shell usage is recommended.
If you would like to apply to attend a workshop, please complete this form.
We will have Duckiebots and Duckietowns for participants to use. However, you are more than welcome to bring your own Duckiebots, available for purchase at https://get.duckietown.com.
We will be hosting three one-day workshops as part of RoboCup 2019 from July 4-6, 2019 for teachers, students, and independent learners who are interested in finding out more about the Duckietown platform. Attendance is completely free and everyone is welcome to apply, even if you are not participating in RoboCup. There are no formal requirements, though basic familiarity with GNU/Linux and shell usage is recommended.
If you would like to apply to attend a workshop, please complete this [form](https://forms.gle/mbAnhgcBRdGBjG2z7).
We will have Duckiebots and Duckietowns for participants to use. However, you are more than welcome to bring your own Duckiebots, available for purchase at [https://get.duckietown.com](https://get.duckietown.com/).
**Categories:** Blog, Events, News
**Tags:** announcement, event, news, robocup, sydney, workshop
---
### [Press Release AI-DO 3](https://duckietown.com/press-release-ai-do2-2/)
**Published:** October 29, 2019
**Author:** Liam Paull
**Content:**
\[pdfviewer width=”600px” height=”849px” beta=”false”\]https://www.duckietown.com/wp-content/uploads/2019/10/AIDO3-Press-release-EN.pdf\[/pdfviewer\]
**Categories:** Uncategorized
---
### [Round 3 of the the AI Driving Olympics is underway!](https://duckietown.com/round-2-of-the-the-ai-driving-olympics-is-underway-2/)
**Published:** October 29, 2019
**Author:** Liam Paull
**Content:**
The AI Driving Olympics (AI-DO) is back!
We are excited to announce the launch of the [AI-DO 3](http://driving-olympics.ai), which will culminate in a live competition event to be held at NeurIPS this Dec. 13-14.
The AI-DO is a global robotics competition that comprises a series of events based on autonomous driving. This year there are three events, urban (Duckietown), advanced perception ([nuScenes](https://www.nuscenes.org/)), and racing ([AWS Deepracer](https://aws.amazon.com/deepracer/)). The objective of the AI-DO is to engage people from around the world in friendly competition, while simultaneously benchmarking and advancing the field of robotics and AI.
Check out our official [press release](https://www.duckietown.com/archives/44563).
- Learn more about the AI-DO competition [here](http://driving-olympics.ai/).
#### If you've already joined the competition we want to hear from you!
Share your pictures on [facebook](https://www.facebook.com/duckietown/) and [twitter](https://twitter.com/DuckietownAI).
**Categories:** Blog, Events, News
**Tags:** AI Driving Olympics, AI-DO, announcement, competition, global, neurips
---
### [AI-DO 3 - Urban Event Winners](https://duckietown.com/ai-do-3-urban-event-winners/)
**Published:** January 14, 2020
**Author:** Jacopo Tani
**Content:**
In case you missed it AI-DO 3 has come and gone. Interested in reliving the competition? Here’s the [video](https://slideslive.com/38922006/competition-track-day-12).
We had a great time at NeurIPS hosting the Third Edition of the AI Driving Olympics. As usual the sound of Duckies attracted an engaging and supportive crowd.
## Racing Event
The competition began with the Racing Event, hosted by **AWS DeepRacer**. They ran their top 10 submissions and selected the winner by who could complete the fastest lap.
Racing Event Winner
Ayrat Baykov at 8:08 seconds
## Advanced Perception Event
The winners of the Advanced Perception Event hosted by **APTIV** and the **nuScenes** dataset were announced. Luckily a member of the winning team was present to accept the award.
Rank 3
CenterTrack – Open and Vision
Rank 2
VV\_Team
Rank 1
StanfordlPRL-TRI
## Urban Event
The competition culminated with **Duckietown’s** own Urban Driving Event, where we ran the top submissions for each of the three challenges on our competition tracks.
*Winners*
Lane Following
JBRRussia1: Konstantin Chaika, Nikita Sazanovich, Kirill Krinkin, Max Kuzmin
Lane Following with Vehicles
phmarm
Lane Following with Vehicles and Intersections
frank\_qcd\_qk
#### Final Scoreboard
#### A few pictures from the event
[](https://duckietown.com/wp-content/uploads/2020/01/20191213_110500.jpg) AWS DeepRacer competition [](https://duckietown.com/wp-content/uploads/2020/01/20191213_111027.jpg) The crowd at AI-DO3 [](https://duckietown.com/wp-content/uploads/2020/01/20191213_111822.jpg) Command central
[](https://duckietown.com/wp-content/uploads/2020/01/20191213_121416.jpg) It is all action in Duckietown [](https://duckietown.com/wp-content/uploads/2020/01/20191213_121509.jpg) Trying to get a glimpse of the action [](https://duckietown.com/wp-content/uploads/2020/01/20191213_121615.jpg) Urban Event – live finals!
[](https://duckietown.com/wp-content/uploads/2020/01/20191213_125235-e1579025485904.jpg) Winner LFVI frank\_qcd\_qk [](https://duckietown.com/wp-content/uploads/2020/01/20191213_125231-e1579025499504.jpg) Winner LFV phmarm [](https://duckietown.com/wp-content/uploads/2020/01/dfteam-aido3.png) Duckietown Team
[](https://duckietown.com/wp-content/uploads/2020/01/team.jpg) The Duckietown Team
Congratulations to all the winners and thanks for participating in the competition. We look forward to seeing you for AI-DO 4!
**Categories:** competition, Research
**Tags:** AI Driving Olympics, AI-DO, competition, neurips, news
---
### [Community Spotlight: Arian Houshmand – Control Algorithms for Traffic](https://duckietown.com/community-spotlight-arian-houshmand-control-algorithms-for-traffic/)
**Published:** January 28, 2020
**Author:** Jacopo Tani
**Content:**
**Boston University, March 7, 2019**: No one likes sitting in traffic: it is a waste of time and damaging to the environment. Thankfully researcher [Arian Houshmand](https://www.linkedin.com/in/arian-houshmand/) from [Boston University CODES lab](https://www.bu.edu/codes/) is on the case, and he’s using Duckietown to help solve the problem.
##### Quick links
- [ Boston University ](https://www.bu.edu/)
- [ Boston University CODES lab ](https://www.bu.edu/codes/)
- [ Arian Houshmand ](https://www.linkedin.com/in/arian-houshmand/)
## Control algorithms to improve traffic
Traffic congestion around the world is worsening, according to transport data firm INRIX. In the U.S. alone, Americans wasted an average of 97 hours in traffic in 2018 – that’s two precious weekends worth of time. Captivity in traffic also costs them nearly $87 billion in 2018, an average of $1,348 per driver. Clearly, the need for smart transportation is reaching a fervor, not only to alleviate the mental and financial state of drivers, but to address the significant economic toll on affected cities.
Traffic congestion around the world is worsening, according to transport data firm INRIX. In the U.S. alone, Americans wasted an average of 97 hours in traffic in 2018 – that’s two precious weekends worth of time. Captivity in traffic also costs them nearly $87 billion in 2018, an average of $1,348 per driver. Clearly, the need for smart transportation is reaching a fervor, not only to alleviate the mental and financial state of drivers, but to address the significant economic toll on affected cities. Fortunately, development of intelligent mobility technologies is advancing. In an ongoing research project funded by the U.S. Department of Energy’s (DOE) Advanced Research Projects Agency-Energy (ARPA-E) NEXTCAR program, BU researchers in collaboration with researchers from University of Delaware, University of Michigan, Oak Ridge National Lab, and Bosch are developing technologies for Connected and Automated Vehicles (CAVs) to increase their fuel efficiency and as a bi-product reduce traffic congestion.
### The goal
The goal of this project is to design control and optimization technologies that enable a plug-in hybrid electric vehicle (PHEV) to communicate with other cars and city infrastructure and act on that information. By providing cars with situational self-awareness, they will be able to efficiently calculate the best possible route, accelerate and decelerate as needed, and manage their powertrain. This is an important task toward advancing the vision to create an ‘Internet of Cars,’ in which connected and self-driving cars operate seamlessly with each other and traffic infrastructure, improving fuel efficiency and safety, and reducing traffic congestion and pollution.
Today’s commercially-available self-driving cars rely on costly sensors, specifically radar, camera, and LIDAR (light) to operate semi-autonomously. In the NEXTCAR project, BU researchers with project collaborators are looking to go beyond that by developing decision-making algorithms to improve the autonomous operation of a single hybrid vehicle as well as algorithms for communications between vehicles and their environment, enabling self-driving cars to cooperate and interact within their socio-cyber-physical environment.
Several different functions have been developed throughout this project including:
● Eco-routing: Procedure of finding the optimal route for a vehicle to travel between two points, which utilizes the least amount of energy costs.
● Eco-AND (Economical Arrival and Departure): An optimal control framework for approaching a traffic light without stopping at the intersection by having traffic light cycle time information.
● CACC (Cooperative Adaptive Cruise Control): An extension of adaptive cruise control
> "We use Duckietown to train students on how to implement their algorithms on embedded systems and also as a means to demonstrate our developed technologies in action and in a live setting."
>
> Arian Houshmand
(ACC) that by benefiting from vehicle to vehicle (V2V) communication increases the safety and energy efficiency by reducing headway.
In order to validate and test the developed technologies, researchers first use simulation environments to test the algorithms. After verifying through simulation, they implement the algorithms on Duckietown, and finally deploy them on real cars (Audi A3 e-tron) at the University of Michigan’s M-city (test track for self-driving cars).
We use Duckietown to train students on how to implement their algorithms on embedded systems and also as a means to demonstrate our developed technologies in action and in a live setting. Since most of our research focuses on Connected and Automated Vehicles (CAVs), we need to establish connections between individual Duckiebots and traffic lights. As a result, we created a platform for exchanging information and control commands between all the cars and traffic lights.
Online localization of Duckiebots is a challenging task, and is missing from the current framework. We relied on our external motion capture sensors (OptiTrack) to localize the robots.
Duckietown is a nice platform for performing experiments on autonomous robots since It is relatively simple to set up the town and Duckiebots. Moreover, the built in perception and lane keeping capabilities are very useful to kick off experiments quickly. Traffic lights and signs are also helpful to create different scenarios for testing algorithms in city-like scenarios.
What would make Duckietown even more useful in our application is feedback sensors for determining wheel rotational speed/position as it is difficult to correct for rotational speed errors of the wheels and a ROS node for exchanging information between robots and traffic lights for testing collaborative control algorithms.
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Find out more use cases ](https://www.duckietown.com/news/people-of-duckietown)
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** People
**Tags:** boston, research, traffic
---
### [Learn robotics from the comfort of your home with Duckietown](https://duckietown.com/stay-at-home-and-distanced-learning/)
**Published:** April 6, 2020
**Author:** Jacopo Tani
**Content:**
These are difficult times for us all.
With physical distancing directives issued across the globe and many people restricted to their homes, we want to reach out (virtually) and offer our support.
To help you beat the isolation blues, the Duckietown Foundation is
[contributing $100](https://get.duckietown.com/discount/STAY@HOME?redirect=%2Fcollections%2Fgetting-started-with-duckietown-hardware)
towards the **next 100 orders** of Duckiebots, Starter Kits, and Navigation Packs.
#### Remember that you can still learn about robotics without a robot!
Almost all of our resources remain open and available for your use.
Join us on [Slack](https://join.slack.com/t/duckietown/shared_invite/enQtNTU0Njk4NzU2NTY1LWM2YzdlNmJmOTg4MzAyODc2YTI3YTc5MzE2MThkZGUwYTFkZWQ4M2ZlZGU1YTZhYjg5YTgzNDkyMzI2ZjNhZWE), peruse our [library](https://docs.duckietown.com/), or start training for the [Urban League of the AI Driving Olympics](https://docs-old.duckietown.org/daffy/AIDO/out/index.html).
Due to the closure of academic institutions the Duckietown Autolabs are temporarily closed.
##### Coming soon: online demonstrations and tutorials to help you get started!
Have fun learning and stay safe!
**Categories:** Uncategorized
---
### [Duckietown and NVIDIA work together for accessible AI and robotics education: Meet the NVIDIA powered Duckiebot](https://duckietown.com/duckietown-and-nvidia-work-together-for-accessible-ai-and-robotics-education-meet-the-nvidia-powered-duckiebot/)
**Published:** October 6, 2020
**Author:** Ivano Marocchi
**Content:**
## Duckietown and NVIDIA partnership for accessible AI and robotics education
NVIDIA GTC, October 6, 2020: Duckietown and NVIDIA align efforts to push the boundaries of accessible, state-of-the-art higher-education in robotics and AI. The tangible outcome is a brand new “Founder’s edition” Duckiebot, which will be broadly available from January 2021, powered by the new NVIDIA Jetson Nano 2GB platform.
Read the full NVIDIA announcement [here](https://blogs.nvidia.com/blog/2020/10/06/jetson-nano-2gb-duckietown).
### Meet the NVIDIA powered Duckiebot
Autonomy is already changing the world. Duckietown and NVIDIA recognize the importance of hands-on education in robotics and AI to empower everybody today to understand and design the next generations of autonomy.
The result of this collaboration is a new NVIDIA powered Duckiebot, using the novel Jetson Nano 2GB board, that will enable local execution of machine learning agents in the Duckietown ecosystem.
To celebrate this special occasion, the Duckiebot has been redesigned to include: new sensors (time of flight, IMU, encoders), a new custom-designed battery providing real time diagnostics (state of charge, remaining autonomy and other health metrics), and fun accessories like a screen to visualize key metrics. All of this while keeping the price accessible for anyone willing to experience the challenges of a real-life robotic ecosystem.
### A great team
“The new NVIDIA Jetson Nano 2GB is the ultimate starter AI computer for educators and students to teach and learn AI at an incredibly affordable price.” said Deepu Talla, Vice President and General Manager of Edge Computing at NVIDIA. “Duckietown and its edX MOOC are leveraging Jetson to take hands-on experimentation and understanding of AI and autonomous machines to the next level.”
“The Duckietown educational platform provides a hands-on, scaled down, accessible version of real world autonomous systems.” said Emilio Frazzoli, Professor of Dynamic Systems and Control, ETH Zurich, “Integrating NVIDIA’s Jetson Nano power in Duckietown enables unprecedented access to state-of-the-art compute solutions for learning autonomy.”
### Learn more
To know more about the technical specifications of the new NVIDIA powered Duckiebot, or to pre-order yours, visit the Duckietown project shop [here](https://get.duckietown.com/products/duckiebot-db21-m/).
The new Duckiebot will be also used in the “Self-driving Cars with Duckietown” Massive Online Open Course (MOOC) that will be held in March 2021 on edX. You can find more information about the MOOC [here](http://www.duckietown.com/mooc/).
**Categories:** Blog, editor-choiche, Media Coverage, News
**Tags:** announcement, duckiebot, NVIDIA
---
### [Robust Reinforcement Learning-based Autonomous Driving Agent for Simulation and Real World](https://duckietown.com/duckietown-and-nvidia-work-together-for-accessible-ai-and-robotics-education-meet-the-nvidia-powered-duckiebot-2-3/)
**Published:** October 7, 2020
**Author:** Jacopo Tani
**Content:**
- [ Title: Robust Reinforcement Learning-based Autonomous Driving Agent for Simulation and Real World - IEEE Conference Publication ](https://ieeexplore.ieee.org/document/9207497)
- Authors: Péter Almási, Róbert Moni, Bálint Gyires-Tóth
- Published: 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, United Kingdom, 2020, pp. 1-8, doi: 10.1109/IJCNN48605.2020.9207497
## Robust Reinforcement Learning-based Autonomous Driving Agent for Simulation and Real World
We asked Róbert Moni to tell us more about his recent work. Enjoy the read!
### The author's perspective
Most of us, proud nerd community members, experience driving first time by the discrete actions taken on our keyboards. We believe that the harder we push the forward arrow (or the W-key), the car from the game will accelerate faster (sooo true 😊 ). Few of us believes that we can resolve this task with machine learning. Even fever of us believes that this can be done accurately and in a robust mode with a basic Deep Reinforcement Learning (DRL) method known as Deep Q-Learning Networks (DQN).
It turned to be true in the case of a Duckiebot, and even more, with some added computer vision techniques it was able to perform well both in simulation (where the training process was carried out) and real world.
### The pipeline
The complete training pipeline carried out in the Duckietown-gym environment is visualized in the figure above and works as follows. First, the camera images go through several preprocessing steps:
- resizing to a smaller resolution (60×80) for faster processing;
- cropping the upper part of the image, which doesn’t contain useful information for the navigation;
- segmenting important parts of the image based on their color (lane markings);
- and normalizing the image;
- finally a sequence is formed from the last 5 camera images, which will be the input of the Convolutional Neural Network (CNN) policy network (the agent itself).
The agent is trained in the simulator with the DQN algorithm based on a reward function that describes how accurately the robot follows the optimal curve. The output of the network is mapped to wheel speed commands.
### The workings
The CNN was trained with the preprocessed images. The network was designed such that the inference can be performed real-time on a computer with limited resources (i.e. it has no dedicated GPU). The input of the network is a tensor with the shape of (40, 80, 15), which is the result of stacking five RGB images. The network consists of three convolutional layers, each followed by ReLU (nonlinearity function) and MaxPool (dimension reduction) operations.
The convolutional layers use 32, 32, 64 filters with size 3 × 3. The MaxPool layers use 2 × 2 filters. The convolutional layers are followed by fully connected layers with 128 and 3 outputs. The output of the last layer corresponds to the selected action. The output of the neural network (one of the three actions) is mapped to wheel speed commands; these actions correspond to turning left, turning right, or going straight, respectively.
### Learn more
Our work was acknowledged and presented at the IEEE World Congress on Computational Intelligence 2020 [ conference](https://wcci2020.org/). We plan to publish the source code after AI-DO5 competition. Our paper is available on [ieeexplore.ieee.org](https://ieeexplore.ieee.org/document/9207497/), deepai.org and [arxiv.org](https://arxiv.org/abs/2009.11212/).
Check out our sim and real [demo on Youtube](https://www.youtube.com/watch?v=rhKoDz6HAsg&feature=emb_title/) performed at our Duckietown Robotarium put together at Budapest University of Technology and Economics. .
**Categories:** paper, Research
**Tags:** paper, reinforcement learning, research, simulation
---
### [The Workshop on Benchmarking Progress in Autonomous Driving at IROS 2020](https://duckietown.com/duckietown-and-nvidia-work-together-for-accessible-ai-and-robotics-education-meet-the-nvidia-powered-duckiebot-3/)
**Published:** October 21, 2020
**Author:** Liam Paull
**Content:**
## The IROS 2020 Workshop on Benchmarking Autonomous Driving
Duckietown has also **a science mission:** to help develop technologies for reproducible benchmarking in robotics.
The **[IROS 2020 Workshop on Benchmarking Autonomous Driving](https://montrealrobotics.ca/driving-benchmarks/0-about.html)** provides a platform to investigate and discuss the methods by which progress in autonomous driving is evaluated, benchmarked, and verified.
**It is free to attend.**
The workshop is structured into 4 panels around four themes.
1. Assessing Progress for the Field of Autonomous Driving
2. How to evaluate AV risk from the perspective of real world deployment (public acceptance, insurance, liability, …)?
3. Best practices for AV benchmarking
4. Algorithms and Paradigms
**The workshop will take place on Oct. 25, 2020 starting at 10am EDT**.
### Invited Panelists
We have a list of excellent invited panelists from academia, industry, and regulatory organizations. These include:
- Emilio Frazzoli (ETH Zürich / Motional)
- Alex Kendall (Wayve)
- Jane Lappin (National Academy of Sciences)
- Bryant Walker Smith (USC Faculty of Law)
- Luigi Di Lillo (Swiss Re Insurance),
- John Leonard (MIT)
- Fabio Bonsignorio (Heron Robots)
- Michael Milford (QUT)
- Oscar Beijbom (Motional)
- Raquel Urtasun (University of Toronto / Uber ATG).
### Please join us...
Please join us on October 25, 2020 starting at 10am EST for what should be a very engaging conversation about the difficult issues around benchmarking progress in autonomous vehicles.
For full details about the event please see [here](https://montrealrobotics.ca/driving-benchmarks/0-about.html).
**Categories:** Blog, Events, News, Research
**Tags:** AVs, IROS, workshop
---
### [IROS2020: Watch The Workshop on Benchmarking Progress in Autonomous Driving](https://duckietown.com/iros2020-watch-the-workshop-on-benchmarking-progress-in-autonomous-driving/)
**Published:** October 28, 2020
**Author:** Ivano Marocchi
**Content:**
## What a start for IROS 2020 with the "Benchmarking Progress in Autonomous Driving" workshop!
The 2020 edition of the International Conference on Intelligent Robots and Systems (IROS) started great with the workshop on “Benchmarking Progress in Autonomous Driving”.
The workshop was held virtually on October 25th, 2020, using an engaging and concise format of a sequence of four 1.5-hour moderated round-table discussions (including an introduction) centered around 4 themes.
The discussions on the methods by which progress in autonomous driving is evaluated, benchmarked, and verified were exciting. Many thanks to all the panelists and the organizers!
Here are the videos of the various sessions.
### Opening remarks
### Theme 1: Assessing progress for the field of autonomous vehicles (AVs)
Moderator: [Andrea Censi](https://www.linkedin.com/in/censi/)
Invited Panelists:
- [Emilio Frazzoli](https://www.linkedin.com/in/emilio-frazzoli-0404ba3/) (ETH Zürich / Motional)
- [Alex Kendall](https://alexgkendall.com/) (Wayve)
- [Jane Lappin](https://www.linkedin.com/in/jane-lappin-b890b512/) (Chair, Transportation Research Board Standing Committee on Vehicle-Highway Automation)
- [Atul Acharya](https://www.linkedin.com/in/atulx/) (Director, AV Strategy at AAA Northern California)
### Theme 2: How to evaluate AV risk from the perspective of real world deployment (public acceptance, insurance, liability, …)?
Moderator: [Jacopo Tani](https://www.linkedin.com/in/jacopo-tani/)
Invited Panelists:
- [Bryant Walker Smith](https://sc.edu/study/colleges_schools/law/faculty_and_staff/directory/smith_bryant_walker.php) (USC Faculty of Law)
- Luigi Di Lillo (Swiss Re Insurance)
- [John Leonard](https://www.csail.mit.edu/person/john-leonard) (MIT)
### Theme 3: Best practices for AV benchmarking
Moderator: [Liam Paull](https://www.linkedin.com/in/liam-paull-83a5442b/)
Invited Panelists:
- [Fabio Bonsignorio](http://www.heronrobots.com/about/people-info/15-fabio-bonsignorio) (Heron Robots)
- [Michael Milford](https://staff.qut.edu.au/staff/michael.milford) (QUT)
- [Oscar Beijbom](https://beijbom.github.io/) (Motional)
- [Marcelo H. Ang Jr](https://www.eng.nus.edu.sg/me/staff/ang-jr-marcelo-h/) (National University of Singapore)
### Theme 4: Do we need new paradigms for AV development?
Moderator: [Matt Walter](https://www.linkedin.com/in/matthew-walter-70320b2/)
Invited Panelists:
- [Raquel Urtasun](http://www.cs.toronto.edu/~urtasun/) (U of Toronto / Uber ATG)
- [Edwin Olson](https://www.linkedin.com/in/edwin-olson/) (May Mobility)
- [Ram Vasudevan](http://www.roahmlab.com/ram-personal) (University of Michigan)
- [Sertac Karaman](http://karaman.mit.edu/) (MIT / Optimus Ride)
### Closing remarks
You can find additional information about the workshop [here](https://montrealrobotics.ca/driving-benchmarks/0-details.html).
**Categories:** Blog, Events, News, Research
---
### [Join the AI Driving Olympics, 5th edition, starting now!](https://duckietown.com/join-the-ai-driving-olympics-5th-edition-starting-now/)
**Published:** November 5, 2020
**Author:** Ivano Marocchi
**Content:**
## Compete in the 5th AI Driving Olympics (AI-DO)
The 5th edition of the Artificial Intelligence Driving Olympics (AI-DO 5) has officially started!
The [AI-DO](https://www.duckietown.com/research/ai-driving-olympics) serves to benchmark the state of the art of artificial intelligence in autonomous driving by providing standardized simulation and hardware environments for tasks related to multi-sensory perception and embodied AI.
Duckietown hosts AI-DO competitions biannually, with finals events held at machine learning and robotics conferences such as the International Conference on Robotics and Automation (ICRA) and the Neural Information Processing Systems (NeurIPS).
The AI-DO 5 will be in conjunction with NeurIPS 2020 and have two leagues: [Urban Driving](#urban) and [Advanced Perception](#perception).
### Urban driving league challenges
This year’s Urban League includes a traditional AI-DO challenge (LF) and introduces two new ones (LFP, LFVM).
#### Lane Following (LF)
[ Get the LF kit ](https://get.duckietown.com/collections/ai-do-kits/products/ai-do-lane-following-lf-challenge-kit)
The most traditional of AI-DO challenges: have a Duckiebot navigate a road loop without intersection, pedestrians (duckies) or other vehicles. The objective is traveling the longest path in a given time while staying in the lane.
#### Lane following with Pedestrian (LFP)
[ Get the LFP kit ](https://get.duckietown.com/collections/ai-do-kits/products/ai-do-lane-following-lf-challenge-kit)
The LFP challenge is new to AI-DO. It builds upon LF by introducing static obstacles (duckies) on the road. The objectives are the same as for lane following, but do not hit the duckies!
#### Lane Following with Vehicles, multi-body (LFVM)
[ Get the LFVM kit ](https://get.duckietown.com/collections/ai-do-kits/products/ai-do-lane-following-with-vehicles-lfv-challenge-kit)
In this traditional AI-DO challenge, contestants seek to travel the longest path in a city without intersections nor pedestrians, but with other vehicles on the road. Except this year there’s a twist. In this year’s novel multi-body variant, all vehicles on the road are controlled by the submission.
### Getting started: the webinars
We offer a short webinar series to guide contestants through the steps for participating: from running our baselines in simulation as well as deploying them on hardware. All webinars are 9 am EST and free!
##### Introduction
Learn about the Duckietown project and the Artificial Intelligence Driving Olympics.
- Nov. 9, 2020
##### ROS baseline
How to run and build upon the “traditional” Robotic Operation System (ROS) baseline.
- Nov. 11, 2020
##### Local development
On the workflow for developing and deploying to Duckiebots, for hardware-based testing.
- Nov. 13, 2020
##### RL baseline
Learn how to use the Pytorch template for reinforcement learning approaches.
- Nov. 16, 2020
##### IL baseline
Introduction to the Tensorflow template, use of logs and simulator for imitation learning.
- Nov. 18, 2020
[ Synch Webinar Calendar ](https://calendar.google.com/calendar/ical/c_e5eb8m6v5f3krbkl3d00ah4at0%40group.calendar.google.com/public/basic.ics)
### Advanced sensing league challenges
Previous AI-DO editions featured: detection, tracking and prediction challenges around the nuScenes dataset.
For the 5th iteration of AI-DO we have a brand new lidar segmentation challenge.
The challenge is based on the recently released lidar segmentation annotations for nuScenes and features an astonishing 1,400,000,000 lidar points annotated with one of 32 labels.
We hope that this new benchmark will help to push the boundaries in lidar segmentation. Please see for more details.
Furthermore, due to popular demand, we will organize the 3rd iteration of the nuScenes 3d detection challenge. Please see for more details.
### AI-DO 5 Finals event
The AI-DO finals will be streamed LIVE during 2020 edition of the Neural Information Processing Systems ([NeurIPS 2020](https://neurips.cc/)) conference in December.
Learn more about the AI-DO [here](https://www.duckietown.com/research/ai-driving-olympics).
[ Participate in AI-DO 5 now ](https://docs-old.duckietown.org/daffy/AIDO/out/index.html)
[ Get your AI-DO hardware kit ](https://get.duckietown.com/collections/ai-do-kits)
[ Join our Slack Community ](https://join.slack.com/t/duckietown/shared_invite/enQtNTU0Njk4NzU2NTY1LWM2YzdlNmJmOTg4MzAyODc2YTI3YTc5MzE2MThkZGUwYTFkZWQ4M2ZlZGU1YTZhYjg5YTgzNDkyMzI2ZjNhZWE)
### Thank you to our generous sponsors!
The Duckietown Foundation is grateful to its sponsors for supporting this fifth edition of the AI Driving Olympics!
[  ](https://motional.com/)
[  ](https://www.swissre.com/)
[  ](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetbot-ai-robot-kit/)
**Categories:** competition, Events, News, Research
**Tags:** AI Driving Olympics, AI-DO, announcement, competition, global, research
---
### [The "Self-Driving cars with Duckietown" Massive Open Online Course on edX](https://duckietown.com/the-self-driving-cars-with-duckietown-hands-on-massive-open-online-course-on-edx/)
**Published:** November 16, 2020
**Author:** Jacopo Tani
**Content:**
## "Self-Driving Cars with Duckietown" hands-on MOOC on edX
We are launching a *massive open online course* (MOOC): [“Self-Driving Cars with Duckietown”](https://www.edx.org/course/self-driving-cars-with-duckietown) on edX, and it is free to attend!
This course is made possible thanks to the support of the **Swiss Federal Institute of Technology in Zurich** (**ETHZ)**, in collaboration with the **University of Montreal**, the **Duckietown Foundation,** and the **Toyota Technological Institute at Chicago**.
This course combines remote and hands-on learning with real-world robots. It is offered on **edX**, the trusted platform for learning, and it is now [open for enrollment](https://www.edx.org/course/self-driving-cars-with-duckietown).
[  ](https://ethz.ch/en.html)
[  ](https://www.umontreal.ca/en/)
[  ](https://www.ttic.edu)
[  ](https://www.edx.org/course/self-driving-cars-with-duckietown)
Learning activities will support the use of Jetson Nano equipped Duckiebots, powered by **NVIDIA**.
[  ](https://blogs.nvidia.com/blog/2020/10/06/jetson-nano-2gb-duckietown/)
[ Enroll on edX ](https://www.edx.org/course/self-driving-cars-with-duckietown)
### Learning autonomy
Participants will engage in software and hardware **hands-on** learning experiences, with focus on overcoming the challenges of deploying autonomous robots in the real world.
This course will explore the theory and implementation of model- and data-driven approaches for making a model self-driving car drive autonomously in an urban environment.
Pedestrian detection
#### MOOC Factsheet
- Name: Self-driving cars with Duckietown
- Start: March 2021
- [ Platform: edX ](https://www.edx.org/course/self-driving-cars-with-duckietown)
- Cost: free to attend
#### Prerequisites
- Basic Linux, Python, Git
- Elements of linear algebra, probability, calculus
- Elements of kinematics, dynamics
- Computer with native Ubuntu installation
#### What you will learn
- Computer Vision
- Robot operations
- Object Detection
- Localization, planning and control
- Reinforcement Learning
[ Enroll on edX ](https://www.edx.org/course/self-driving-cars-with-duckietown)
### Why Self-driving cars with Duckietown?
Teaching **autonomy** requires a fundamentally different approach when compared to other computer science and engineering disciplines, because it is **multi-disciplinary**. Mastering it requires expertise in domains ranging from fundamental mathematics to practical machine-learning skills.
Robot Perception
Robots operate in the real world, and **theory and practice often do not play well together**. There are many hardware platforms and software tools, each with its own strengths and weaknesses. It is not always clear what **tools** are worth investing time in mastering, and how these **skills** will **generalize** to different platforms.
Duckiebot Detection
### Learning through challenges
Progressing through **behaviors of increasing complexity**, participants uncover concepts and tools that address the limitations of previous approaches. This allows to get Duckiebots to actually do things, while gradually **re-iterating concepts** through **various technical frameworks**. Simulation and real-world experiments will be performed using a **Python**, **ROS**, and **Docker** based software stack.
Robot Planning
[ Enroll on edX ](https://www.edx.org/course/self-driving-cars-with-duckietown)
## (Hidden) This line and everything under this line are hidden
This course combines remote and hands-on learning with real-world robots.
It is offered on **edX**, the trusted platform for learning, and it is now open for enrollment.
Learning activities will support the use of **NVIDIA** Jetson Nano powered Duckiebots.
**Categories:** Classes, education, Events
**Tags:** edx, MOOC, NVIDIA, ttic, udm
---
### [Imitation Learning Approach for AI Driving Olympics Trained on Real-world and Simulation Data Simultaneously](https://duckietown.com/imitation-learning-approach-for-ai-driving-olympics-trained-on-real-world-and-simulation-data-simultaneously/)
**Published:** December 4, 2020
**Author:** Konstantin Chaika
**Content:**
- Imitation Learning Approach for AI Driving Olympics Trained on Real-world and Simulation Data Simultaneously
- Mikita Sazanovich, Konstantin Chaika, Kirill Krinkin, Aleksei Shpilman
- [ Workshop on AI for Autonomous Driving (AIAD), the 37th International Conference on Machine Learning, Vienna, Austria, 2020 ](https://sites.google.com/view/aiad2020/accepted-papers)
- [ ArXiv version download: arXiv:2007.03514 ](https://arxiv.org/pdf/2007.03514)
- [ Find code here ](https://github.com/duckietown/challenge-aido_LF-baseline-JBR)
## Imitation Learning Approach for AI Driving Olympics Trained on Real-world and Simulation Data Simultaneously
The AIDO challenge is divided into two global stages: simulation and real-world. A single algorithm needs to perform well in both. It was quickly identified that one of the major problems is the simulation to real-world transfer.
Many algorithms trained in the simulated environment performed very poorly in the real world, and many classic control algorithms that are known to perform well in a real-world environment, once tuned to that environment, do not perform well in the simulation. Some approaches suggest randomizing the domain for the simulation to real-world transfer.
We propose a novel method of training a neural network model that can perform well in diverse environments, such as simulations and real-world environment.
### Dataset Generation
To that end, we have trained our model through imitation learning on a dataset compiled from four different sources:
1. Real-world Duckietown dataset from logs.duckietown.com (REAL-DT).
2. Simulation dataset on a simple loop map (SIM-LP).
3. Simulation dataset on an intersection map (SIM-IS).
4. Real-world dataset collected by us in our environment with car driven by PD controller (REAL-IH).
We aimed to collect data with as many possible situations such as twists in the road, driving in circles clockwise/counterclockwise, and so on. We have also tried to diversify external factors such as scene lighting, items in the room that can get into the camera’s field of view, roadside objects, etc. If we keep these conditions constant, our model may overfit to them and perform poorly in a different environment. For this reason, we changed the lighting and environment after each duckiebot run. The lane detection was calibrated for every lighting condition since different lighting changes the color scheme of the image input.
We made the following change to the standard PD algorithm: since most Duckietown turns and intersections are standard-shaped, we hard-coded the robot’s motion in these situations, but we did not exclude imperfect trajectories. For example, the ones that would go slightly out of bounds of the lane. Imperfections in the robot’s actions increase the robustness of the model.
### Neural network architecture and training
Original images are 640×480 RGB. As a preprocessing step, we remove the top third of the image, since it mostly contains the sky, resize the image to 64×32 pixels and convert it into the YUV colorspace.
We have used 5 convolutional layers with a small number of filters, followed by 2 fully-connected layers. The small size of the network is not only due to it being less prone to overfitting, but we also need a model that can run on a single CPU on RaspberryPi.
We have also incorporated [Independent-Component (IC) layers](https://arxiv.org/pdf/1905.05928.pdf). These layers aim to make the activations of each layer more independent by combining two popular techniques, BatchNorm and Dropout. For convolutional layers, we substitute Dropout with Spatial Dropout which has been shown to work better with them. The model outputs two values for voltages of the left and the right wheel drives. We use the mean square error (MSE) as our training loss.
### Results
For the training evaluation, we compute the mean square error (MSE) of the left and the right wheels outputs on the validation set of each data source.
The first table shows the results for the models trained on all data sources (HYBRID), on real-world data sources only (REAL) and on simulation data sources only (SIM). As we can see, while training on a single dataset sometimes achieves lower error on the same dataset than our hybrid approach. We can also see that our method performs on par with the best single methods. In terms of the average error it outperforms the closest one tenfold. This demonstrates definitively the high dependence of MSE on the training method, and highlights the differences between the data sources.
The next table shows simulation closed-loop performance for all our approaches using the Duckietown simulator. All methods drove for 15 seconds without major infractions, and the SIM model that was trained specifically on the simulation data only drove just 1.8 tiles more than our hybrid approach.
The third table shows the closed-loop performance in the real-world environment. Comparing the number of tiles, we see that our hybrid approach drove about 3.5 tiles more than the following in the rankings model trained on real-world data only.
### Conclusion
Our method follows the imitation learning approach and consists of a convolutional neural network which is trained on a dataset compiled from data from different sources, such as simulation model and real-world Duckietown vehicle driven by a PD controller, tuned to various conditions, such as different map configuration and lighting.
We believe that our approach of emphasizing neurons independence and monitoring generalization performance can offer more robustness to control models that have to perform in diverse environments. We also believe that the described approach of imitation learning on data obtained from several algorithms that are fitted to specific environments may yield a single algorithm that will perform well in general.
—
JBRRussia1 team
[ Join the AI-DO! ](https://www.duckietown.com/research/AI-Driving-olympics)
**Categories:** paper, Research
---
### [AI-DO 5 competition Update](https://duckietown.com/ai-do-update/)
**Published:** December 5, 2020
**Author:** Andrea Censi
**Content:**
## AI-DO 5 Update
AI-DO 5 is in full swing and we want to bring you some updates: better graphics, more maps, faster and more reliable backend and an improved GUI to submit to challenges!
### Challenges visualization
We **updated the visualization**. Now the evaluation produces **videos with your name** and evaluation number (as below).
### Challenges updates
We **fixed** some of the **bugs in the simulator** regarding the visualization (“phantom robots” popping in and out).
We **updated the maps** in the challenges to have more variety in the road network; we put more grass and trees to make the maps more joyful!
 We have updated the maps with more trees and grass
### Faster and more reliable backend
The server was getting slow given the number of submissions, and sometime the service was unavailable. We have **revamped the server code** and **added some backend capacity** to be more fault-tolerant. It is now much faster!
Thanks so much to the participants that helped us debug this problem!
 We overhauled the server code to make it much faster!
### More evaluators
We brought online many more CPU and GPU evaluators. We now **encourage** **you to submit more often** as we have a lot more capacity.
 We have many more evaluators now!
### Submit to testing challenges
We also remind you that the challenges on the front page are the **validation challenges,** in which everybody can see the output. However what counts for winning are the **testing challenges**!
To do that you can use dts challenges submit with the –challenges option
Or, you can use a new way using the website that we just implemented, described below.
#### Submitting to other challenges
**Step 1:** Go to your user page, by clicking “login” and then going to “My Submissions”.
**Step 2:** In this page you will find your submissions grouped by “component”.
Click the component icon as in the figure.
**Step 3:** The page will contain some buttons that allow you to submit to other challenges that you didn’t submit to yet.
[ Join the AI Driving Olympics ](https://www.duckietown.com/research/AI-Driving-olympics)
**Categories:** Blog, News
**Tags:** AI Driving Olympics, challenges-server, infrastructure, technical update
---
### [AI-DO 5 competition leaderboard update](https://duckietown.com/ai-do-5-leaderboard-update/)
**Published:** December 9, 2020
**Author:** Jacopo Tani
**Content:**
## AI-DO 5 pre-finals update
With the fifth edition of the AI Driving Olympics finals day approaching, **1326 solutions** submitted from **94 competitors** in three challenges, it is time to glance over [at the leaderboards](https://challenges.duckietown.com/v4/)!
### Leaderboards updates
This year’s challenges are **lane following** (`LF`), **lane following with pedestrians** (`LFP`) and **lane following with other vehicles**, **multibody** (`LFV_multi`). Learn more about the challenges [here](https://docs-old.duckietown.org/daffy/AIDO/out/aido_rules.html). Each submission can be sent to multiple challenges. Let’s look at some of the most promising or interesting submissions.
#### The Montréal menace
**Raphael Jean** at Mila / University of Montréal is a new entrant for this year.
An interesting submission: [submission #12962 ](https://challenges.duckietown.com/v4/humans/submissions/12962)
[All of raph’s submissions](https://challenges.duckietown.com/v4/humans/users/2565).
#### The submissions from the cold
Team **[JetBrains](http://jetbrains.com)** from Saint Petersburg was a winner of previous editions of AI-DO. They have been dominating the leaderboards also this year.
Interesting submissions: [submission #12905](https://challenges.duckietown.com/v4/humans/submissions/12905)
All of JetBrains submissions: [JBRRussia1. ](https://challenges.duckietown.com/v4/humans/users/2096)
#### BME Conti
PhD student Robert Moni (BME-Conti) from Hungary.
Interesting submissions: [submission #12999 ](https://challenges.duckietown.com/v4/humans/submissions/12999)
All submissions: [timur-BMEconti](https://challenges.duckietown.com/v4/humans/users/2833)
### Deadline for submissions
The deadline for submitting to the AI-DO 5 is **12am EST on Thursday, December 10th, 2020**. The top three entries (more if time allows) in each simulation challenge will be evaluated on real robots and presented at [the finals event at NeurIPS 2020](https://neurips.cc/virtual/2020/public/workshop_19593.html), which happens at **5pm EST on Saturday, December 12**.
[
## Join the AI Driving Olympics now!
Test your skills in the urban league challenges and qualify for the finals at NeurIPS!
Join the AI-DO
AI-DO 5
](https://docs.duckietown.com/daffy/AIDO/out/quickstart.html)
**Categories:** Blog, Events, News, Research
**Tags:** AI Driving Olympics, aido5
---
### [AI Driving Olympics 5th edition: results](https://duckietown.com/ai-do-5-leaderboard-update-2-2/)
**Published:** December 19, 2020
**Author:** Ivano Marocchi
**Content:**
## AI-DO 5: Urban league winners
This year’s challenges were **lane following** (`LF`), **lane following with pedestrians** (`LFP`) and **lane following with other vehicles**, **multibody** (`LFV_multi`).
Let’s find out the results in each category:
### LF
1. **Andras Beres 🇭🇺**
2. Zoltan Lorincz 🇭🇺
3. András Kalapos 🇭🇺
### LFP
1. **Bea Baselines 🐤**
2. Melisande Teng 🇨🇦
3. Raphael Jean 🇨🇦
### LFV\_multi
1. **Robert Moni** 🇭🇺
2. Márton Tim 🇭🇺
3. Anastasiya Nikolskay 🇷🇺
Congratulations to the Hungarian Team from the Budapest University of Technology and Economics for collecting the highest rankings in the urban league!
Here’s how the winners in each category performed both in the qualification (simulation) and in the finals running on real hardware:
#### Andras Beres - Lane following (LF) winner
[ See the evaluation details ](https://challenges.duckietown.com/v4/humans/submissions/13277)
#### Melisande Teng - Lane following with pedestrians (LFP) winner
[ See the evaluation details ](https://challenges.duckietown.com/v4/humans/submissions/13305)
#### Robert Moni - Lane following with other vehicles, multibody (LFV\_multi) winner
[ See the evaluation details ](https://challenges.duckietown.com/v4/humans/submissions/13309)
### AI-DO 5: Advanced Perception league winners
Great participation and results in the Advanced Perception league! Check out this year’s winners in the video below:
### AI-DO 5 sponsors
Many thanks to our amazing sponsors, without which none of this would have been possible!
Stay tuned for next year AI Driving Olympics. Visit the AI-DO page for more information on the competition and to browse this year’s introductory webinars, or check out the Duckietown massive open online course (MOOC) and prepare for next year’s competition!
[ Learn more about the MOOC ](https://www.duckietown.com/mooc)
[ Learn more about the AI-DO ](https://www.duckietown.com/research/AI-Driving-olympics)
**Categories:** Events, Research
**Tags:** AI Driving Olympics, aido5
---
### [“Self-Driving Cars with Duckietown” MOOC starting soon](https://duckietown.com/self-driving-cars-with-duckietown-2020-mooc-starting-soon/)
**Published:** March 4, 2021
**Author:** Duckietown Admin
**Content:**
## Join the first hardware based MOOC about autonomy on edX!
Are you curious about robotics, self-driving cars, and want an opportunity to build and program your own? Set to start on **March 22nd**, **2020, “Self-Driving Cars with Duckietown”** is a hands-on introduction to vehicle autonomy, and the first ever self-driving cars MOOC with a hardware track!
Designed for university-level students and professionals, this course is brought to you by the **Swiss Federal Institute of Technology in Zurich (ETHZ)**, in collaboration with the **University of Montreal**, the **Duckietown Foundation**, and the **Toyota Technological Institute at Chicago**.
[  ](https://ethz.ch/en.html)
**Learning autonomy** requires a **fundamentally different approach** when compared to other computer science and engineering disciplines. Autonomy is inherently **multi-disciplinary**, and mastering it requires expertise in domains ranging from fundamental mathematics to practical machine-learning skills.
[  ](https://www.umontreal.ca/en/)
[  ](https://www.ttic.edu)
This course will explore the theory and implementation of model- and data-driven approaches for making a **model self-driving car drive autonomously** in an urban environment, while **detecting and avoiding pedestrians** (rubber duckies)!
[  ](https://www.edx.org/course/self-driving-cars-with-duckietown)
[ Enroll for free on edX ](https://www.edx.org/course/self-driving-cars-with-duckietown)
In this course you will learn, hands-on, introductory elements of:
- computer vision
- robot operations
- ROS, Docker, Python, Ubuntu
- autonomous behaviors
- modelling and control
- localization
- planning
- object detection and avoidance
- reinforcement learning.
[  ](https://www.duckietown.com/mooc)
The Duckietown robotic ecosystem was created at the **MIT** Computer Science and Artificial Intelligence Laboratory (CSAIL) in 2016 and is now used in over 90 universities worldwide.
“The Duckietown educational platform provides a hands-on, scaled down, accessible version of real world autonomous systems.” said **Emilio Frazzoli**, **Professor of Dynamic Systems and Control, ETH Zurich**, “Integrating NVIDIA’s Jetson Nano power in Duckietown enables unprecedented access to state-of-the-art compute solutions for learning autonomy.”
[ Enroll for free on edX ](https://www.edx.org/course/self-driving-cars-with-duckietown)
This massive online open course will be have a **hands-on learning approach** using, for the hardware track, real robots. You will learn how autonomous vehicles make their own decisions, going from theory to implementation, deployment in simulation as well as on the new **NVIDIA Jetson Nano** powered **Duckiebots**.
“The new NVIDIA Jetson Nano 2GB is the ultimate starter AI computer for educators and students to teach and learn AI at an incredibly affordable price.” said **Deepu Talla, Vice President and General Manager of Edge Computing at NVIDIA**. “Duckietown and its edX MOOC are leveraging Jetson to take hands-on experimentation and understanding of AI and autonomous machines to the next level.”
[  ](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetbot-ai-robot-kit/)
The Duckiebot MOOC Founder’s edition kits are available worldwide, and thanks to **OKdo**, are now available with **free shipping** in the **United States** and in **Asia**!
[ Get the Hardware in the USA or Asia ](#)
[ Get the Hardware in the rest of the world ](#)
“I’m thrilled that ETH, with UMontreal, the Duckietown Foundation, and the Toyota Technological Institute in Chicago, are collaborating to bring this course in self-driving cars and robotics to the 35 million learners on edX. This emerging technology has the potential to completely change the way we live and travel, and the course provides a unique opportunity to get in on the ground floor of understanding and using the technology powering autonomous vehicles,” said **Anant Agarwal, edX CEO and Founder, and MIT Professor**.
Enroll now and don’t miss the chance to join in the first vehicle autonomy MOOC with hands-on learning!
[ Enroll for free on edX ](https://www.edx.org/course/self-driving-cars-with-duckietown)
**Categories:** education, Events, News
**Tags:** DB21, MOOC, teaching
---
### [EdTech awards 2021: Duckietown finalist in 3 categories!](https://duckietown.com/edtech-2021-duckietown-finalist-in-3-categories/)
**Published:** July 1, 2021
**Author:** Ivano Marocchi
**Content:**
## Duckietown reaches the finals in the EdTech Awards 2021
The [EdTech awards](https://www.edtechdigest.com/the-edtech-awards/ "Education Technologies Awards") are the largest and most competitive recognition program in all of education technology.
The competition, led by the EdTech digest, recognizes the biggest names in edtech – and those who soon will be, by identifying all over the world the products, services and people that bet promote education through the use of technology, for the benefit of learners.
[The 2021 edition](https://www.edtechdigest.com/2021-finalists-winners/ "Edtech Awards 2021 edition") has brought a big surprise to Duckietown, as it was nominated as a finalist in 3 different categories:
- **Cool Tool Award**: as robotics (for learning, education) solution;
- **Cool Tool Award**: as higher education solution;
- **Trendsetter Award**: as a product or service setting a trend in education technologies.
Although a final is just a starting point, we are proud of the hard work done by the team in this particularly difficult year of pandemic and lockdowns, and grateful to you all for the incredible support, constructive feedback and contributions!
 To the future, and beyond!
### (hidden) Want to learn more about us?
[ Click here ](https://www.duckietown.com/about/duckietown-foundation)
**Categories:** editor-choiche, Media Coverage, News
**Tags:** competition, education, news, outreach, stem
---
### [Embedded out-of-distribution detection on an autonomous robot platform](https://duckietown.com/embedded-out-of-distribution-detection-on-an-autonomous-robot-platform/)
**Published:** July 13, 2021
**Author:** michaelj004
**Content:**
# Embedded out-of-distribution detection on an autonomous robot platform
- Embedded out-of-distribution detection on an autonomous robot platform
- Michael Yuhas, Yeli Feng, Daniel Jun Xian Ng, Zahra Rahiminasab, Arvind Easwa M ran
- [ Design Automation for CPS and IoT (DESTION 2021) Workshop ](https://cps-vo.org/)
- [ ACM Digital Library ](https://dl.acm.org/doi/abs/10.1145/3445034.3460509)
- [ Code available here ](https://github.com/CPS-research-group/CPS-NTU-Public/tree/DESTION2021)
- [ Data available here ](https://researchdata.ntu.edu.sg/dataset.xhtml?persistentId=doi:10.21979/N9/FVVHNK)
## Introduction
Machine learning is becoming more and more common in cyber-physical systems; many of these systems are safety critical, e.g. autonomous vehicles, UAVs, and surgical robots. However, machine learning systems can only provide accurate outputs when their input data is similar to their training data. For example, if an object detector in an autonomous vehicle is trained on images containing various classes of objects, but no ducks, what will it do when it encounters a duck during runtime? One method for dealing with this challenge is to detect inputs that lie outside the training distribution of data: out-of-distribution (OOD) detection. Many OOD detector architectures have been explored, however the cyber-physical domain adds additional challenges: hard runtime requirements and resource constrained systems. In this paper, we implement a real-time OOD detector on the Duckietown framework and use it to demonstrate the challenges as well as the importance of OOD detection in cyber-physical systems.
## Out-of-Distribution Detection
Machine learning systems perform best when their test data is similar to their training data. In some applications unreliable results from a machine learning algorithm may be a mere nuisance, but in other scenarios they can be safety critical. OOD detection is one method to ensure that machine learning systems remain safe during test time. The goal of the OOD detector is to determine if the input sample is from a different distribution than that of the training data. If an OOD sample is detected, the detector can raise a flag indicating that the output of the machine learning system should not be considered safe, and that the system should enter a new control regime. In an autonomous vehicle, this may mean handing control back to the driver, or bringing the vehicle to a stop as soon as practically possible.
In this paper we consider the existing β-VAE based OOD detection architecture. This architecture takes advantage of the information bottleneck in a variational auto-encoder (VAE) to learn the distribution of training data. In this detector the VAE undergoes unsupervised training with the goal of minimizing the error between a true prior probability in input space *p(z)*, and an approximated posterior probability from the encoder output *p(z|x)*. During test time, the Kullback-Leibler divergence between these distributions *p(z)* and *q(z|x)* will be used to assign an OOD score to each input sample. Because the training goal was to minimize the distance between these two distributions on in-distribution data, in-distribution data found at runtime should have a low OOD score while OOD data should have a higher OOD score.
## Duckietown
We used Duckietown to implement our OOD detector. Duckietown provides a natural test bed because:
- *It is modular and easy to learn*: the focus of our research is about implementing an OOD detector, not building a robot from scratch
- *It is a resource constrained system*: the RPi on the DB18 is powerful enough to be capable of navigation tasks, but resource constrained enough that real-time performance is not guaranteed. It servers as a good analog for a system in which an OOD detector shares a CPU with perception, planning, and control software.
- *It is open source*: this eliminates the need to purchase and manage licenses, allows us to directly check the source code when we encounter implementation issues, and allows us to contribute back to the community once our project is finished.
- *It is low-cost*: we’re not made of money 🙂
In our experiment, we used the stock DB18 robot. Because we took advantage of the existing Duckietown framework, we only had to write three ROS nodes ourselves:
- *Lane following node*: a simple OpenCV-based lane follower that navigates based on camera images. This represents the perception and planning system for the mobile robot that we are trying to protect. In our system the lane following node takes 640×480 RGB images and updates the planned trajectory at a rate of 5Hz.
- *OOD detection node*: this node also takes images directly from the camera, but its job is to raise a flag when an OOD input appears (image with an OOD score greater than some threshold). On the RPi with no GPU or TPU, it takes a considerable amount of time to make an inference on the VAE, so our detection node does not have a target rate, but rather uses the last available camera frame, dropping any frames that arrive while the OOD score is being computed.
- *Motor control node*: during normal operation it takes the trajectory planned by the lane following node and sends it to the wheels. However, if it receives a signal from the OOD detection node, it begins emergency breaking.
## The Experiment
Our experiment considers the emergency stopping distance required for the Duckiebot when an OOD input is detected. In our setup the Duckiebot drives forward along a straight track. The area in front of the robot is divided into two zones: the risk zone and the safe zone. The risk zone is an area where if an obstacle appears, it poses a risk to the Duckiebot. The safe zone is further away and to the sides; this is a region where unknown obstacles may be present, but they do not pose an immediate threat to the robot. An obstacle that has not appeared in the training set is placed in the safe zone in front of the robot. As the robot drives forward along the track, the obstacle will eventually enter the risk zone. Upon entry into the risk zone we measure how far the Duckiebot travels before the OOD detector triggers an emergency stop.
We defined the risk zone as the area 60cm directly in front of our Duckiebot. We repeated the experiment 40 times and found that with our system architecture, the Duckiebot stopped on average 14.5cm before the obstacle. However, in 5 iterations of the experiment, the Duckiebot collided with the stationary obstacle.
We wanted to analyze what lead to the collision in those five cases. We started by looking at the times it took for our various nodes to run. We plotted the distribution of end-to-end stopping times, image capture to detection start times, OOD detector execution times, and detection result to motor stop times. We observed that there was a long tail on the OOD execution times, which lead us to suspect that the collisions occurred when the OOD detector took too long to produce a result. This hypothesis was bolstered by the fact that even when a collision had occurred, the last logged OOD score was above the detection threshold, it had just been produced too late. We also looked at the final two OOD detection times for each collision and found that in every case the final two times were above the median detector execution time. This highlights the importance of real-time scheduling when performing OOD detection in a cyber-physical system.
We also wanted to analyze what would happen if we adjusted the OOD detection threshold. Because we had logged the the detection threshold every time the detector had run, we were able to interpolate the position of the robot at every detection time and discover when the robot would have stopped for different OOD detection thresholds. We observe there is a tradeoff associated with moving the detection threshold. If the detection threshold is lowered, the frequency of collisions can be reduced and even eliminated. However, the mean stopping distance is also moved further from the obstacle and the robot is more likely to stop spuriously when the obstacle is outside of the risk zone.
## Next Steps
In this paper we successfully implemented an OOD detector on a mobile robot, but our experiment leaves many more questions:
- How does the performance of other OOD detector architectures compare with the β-VAE detector we used in this paper?
- How can we guarantee the real-time performance of an OOD detector on a resource-constrained system, especially when sharing a CPU with other computationally intensive tasks like perception, planning, and control?
- Does the performance vary when detecting more complex OOD scenarios: dynamic obstacles, turning corners, etc.?
### Did you find this interesting?
Read more Duckietown based papers [here](https://www.duckietown.com/research/papers).
**Categories:** paper, Research
---
### [Automatic Wheels and Camera Calibration for Monocular and Differential Mobile Robots](https://duckietown.com/automatic-wheels-and-camera-calibration-for-monocular-and-differential-mobile-robots/)
**Published:** July 21, 2021
**Author:** Konstantin Chaika
**Content:**
- Automatic Wheels and Camera Calibration for Monocular and Differential Mobile Robots
- Konstantin Chaika, Anton Filatov, Artyom Filatov, Kirill Krinkin
- [ Applied Sciences 11, no. 13: 5806. ](https://doi.org/10.3390/app11135806)
## Automatic Wheels and Camera Calibration for Monocular and Differential Mobile Robots
After assembling the robot, components such as the camera and wheels need to be calibrated. This requires human participation and depends on human factors. We describe the approach to fully automatic calibration of a robot’s camera and wheels.
The camera calibration collects the necessary set of images by automatically moving the robot in front of the chess boards, and then moving it on the marked floor, assessing its trajectory curvature. As a result of the calibration, coefficient k is calculated for the wheels, and camera matrix K (which includes the focal length, the optical center, and the skew coefficient) and distortion coefficients D are calculated for the camera.
Proposed approach has been tested on duckiebots in Alexander Popov’s International Innovation Institute for Artificial Intelligence, Cybersecurity and Communication, SPbETU “LETI”. This solution is comparable to manual calibrations and is capable of replacing a human for this task.
### Camera calibration process
The initial position of the robot is a part of the floor with chessboards in front, where the robot is located from the very beginning, on which its camera is directed and the floorsurface is marked with aruco markers on the other side of it.
There can be any number of chessboards, determined by the amount of free space around the robot. To a greater extent, the accuracy of calibration is affected by the frames with different positions of the boards, e.g., boards located at different distances from the robot and at different angles. The physical size and type of all the boards around the robot must be the same.
In fact, the camera calibration implies that the robot is rotating around its axis and taking pictures of all the viewable chessboards in turn. In this case, the ability to make several “passes” during the shooting process should be provided for, to control which of the boards the robot is currently observing and in which direction it should turn. As a result, the algorithm can be represented as a sequence of actions: “get a frame from the camera” and “turn” a little. The final algorithm comprises the following sequence of actions:
1. Obtain frame from the camera;
2. Find a chessboard on the camera frame;
3. Save information about board corners found in the image;
4. Determine the direction of rotation according to the schedule;
5. Make a step;
6. Either repeat the steps described above, or complete the data
collection and proceed with the camera calibration using OpenCV.
### Wheels calibration process
Floor markers should be oriented towards the chessboards and begin as close to the robot as possible. The distance between the markers depends on camera’s resolution, as well as its height and angle of inclination, but it must be such that at least three recognizable markers can simultaneously be in the frame. For ours experiments, the distance between the markers was set as 15 cm with a marker size of 6.5 cm. The algorithm does not take into account the relative position of the markers against each other; however, the orientation of all markers must be strictly the same.
Let us consider the first iteration of the automatic wheel calibration algorithm:
1. The robot receives the orientation of the marker closest to it and remembers it.
2. Next, the robot moves forward with thespeeds of the left and right wheels equal to
ω1ω2 for some fixed time t. The speeds are calculated taking into account the calibration
coefficient k, which for the first iteration is chosen to equal 1 – that is, it is assumed that
the real wheel speeds are equal.
3. The robot obtains the orientation of the marker closest to it again and calculates the
difference in angles between them.
4. The coefficient ki for this step is calculated.
5. The robot moves back for the same time t.
In order to reduce the influence of the error in calculating ki, coefficient k is refined only by the value of (ki−1)/2 after each iteration. It is important to complete this step after the robot moves back, because it reduces the chance of the robot moving outside the area width. If, after the next step, the modulus of the difference between (ki−1)/2 and 1.0 becomes less than the pre-selected E, then at this iteration (ki−1)/2 is not taken into account. If after three successive iterations ki is not taken into account, the wheel calibration is considered to be completed.
### Accuracy Evaluation
To compare camera calibration errors, the knowledge of how to calculate these errors is needed. Since the calibration mechanism is used by the OpenCV library, the error is also calculated by the method offered by this library.
As noted earlier, with respect to calibration factors, the approach used to calibrate the camera is not applicable. Therefore, the influence of the coefficient on the robot’s trajectory curvature is estimated. To do this, the robot was located at a certain fixed distance from a straight line, along which it was oriented and then moved in manual mode strictly directly to a distance of two meters from the start point along the axis, relative to which it was oriented. Then, the robot stopped and the distance between the initial distance to the line and the final one was calculated.
Two metrices were estimated – reprojection error and straight line deviation. First one shows the quality of camera calibration, and the second one represents the quality of wheels calibration. Two pictures below present result of 10 independent tests in comparison with manual calibration.
The tests found that the suggested solution, on average, shows that the results are not much worse, than the classical manual solution when calibrating the camera, as well as when calibrating the wheels with a well known calibrated camera. However, when calibrating both the wheels and the camera, the wheel calibration can be significantly affected by the camera calibration effect. As a result of testing, a clear relationship was found between the reprojection error and the straight line deviation.
### Method Modifications
After the integration of this approach, it became necessary to automate the last step-moving the robot to the field. Due to the fact that after the calibration step completion the robot becomes fully prepared for launching autonomous driving algorithms on it, the automation of this step further reduces the time spent by the operator when calibrating the robot, since instead of moving the robot to the field manually, he can place the next robot at the starting position. In our case, the calibration field was located at the side of the road lane so that the floor markers used to calibrate the wheels are oriented perpendicular to the road lane.
Thus, the first stage of the robot automatic removal from the calibration zone is to return its orientation back to the same state, as it was at the moment when the wheel calibration started. This was carried out using exactly the same approach that was described earlier—depending on the orientation of the floor marker closest to the robot, the robot rotates step by step about its axis clockwise or counterclockwise until the value of the robot’s orientation angle is modulo less than some preselected value.
At this point, the robot is still on the wheel calibration field, but in this case, it is oriented towards the lane. Thus, the last step is to move the robot outside the border of the field with markers. To do this, it is enough to give the robot a command to move directly until it stops observing the markers, when the last marker is hidden from the camera view. This means that the robot has left the calibration zone, and the robot can be put into the lane following mode.
### Future Work
During the robot’s operation, the wheels calibration may become irrelevant. It can be influenced by various factors: a change in the wheel diameter due to wear of the wheel coating, a slight change in the characteristics of motors due to the wear of the gearbox plastic, and a change in the robot’s weight distribution, e.g., laying the cables on the other side of the case after charging the robot, and so a slight calibration mismatch can occur. However, all these factors have a rather small impact, and the robot will still have a satisfactory calibration. There is no need to re-perform the calibration process, just a little refinement of the current one seems to be enough. To do this, a section of the road along which the robots will be guaranteed to pass regularly, was selected.
Further, markers were placed in this lane according to the rules described earlier: the distance between the markers is 15 cm; the size of the marker is 6.5 cm. The markers are located in the center of the lane. The distance between the markers may be not completely accurate, but they should be oriented in the same direction and co-directed with the movement in the lane on which they are placed.
The first marker in the direction of travel must have a predefined ID. It can be anything, the only limitation is that it must be unique for a current robot environment. Further, the following changes were made to the algorithm for the standard control of the robot: when the robot recognizes the first marker with a predetermined ID while driving right in the lane, it corrects its orientation relative to this marker and continues to move strictly straight ahead. Further, the algorithm is similar to the one described earlier—the robot recognizing the next marker can refine its wheel calibration coefficient, apply it, and change the orientation coaxially with the next marker.
### Conclusions
As a result, a solution was developed that allows a fully automatic calibration of the camera and the Duckiebot’s wheels. The main feature is the autonomy of the process, which allows one person to run the calibration of an arbitrary number of robots in parallel and not be blocked during their calibration. In addition, the robot is able to improve its calibration as it operates in default mode.
Comparing the developed solution with the initial one resulted in finding a slight deterioration in accuracy, which is primarily associated with the accuracy of the camera calibration; however, the result obtained is sufficient for the robot’s initial calibration and is comparable to manual calibration.
### Did you find this interesting?
Read more Duckietown based papers [here](https://www.duckietown.com/research/papers).
**Categories:** paper, Research
---
### [Join the AI Driving Olympics, 6th edition, starting now!](https://duckietown.com/join-the-ai-driving-olympics-5th-edition-starting-now-2/)
**Published:** November 1, 2021
**Author:** Ivano Marocchi
**Content:**
# The 2021 AI Driving Olympics
Compete in the 2021 edition of the Artificial Intelligence Driving Olympics (AI-DO 6)!
The [AI-DO](https://www.duckietown.com/research/ai-driving-olympics) serves to benchmark the state of the art of artificial intelligence in autonomous driving by providing standardized simulation and hardware environments for tasks related to multi-sensory perception and embodied AI.
Duckietown traditionally hosts AI-DO competitions biannually, with finals events held at machine learning and robotics conferences such as the International Conference on Robotics and Automation (ICRA) and the Neural Information Processing Systems (NeurIPS).
AI-DO 6 will be in conjunction with [NeurIPS 2021](https://nips.cc/Conferences/2021/CompetitionTrack) and have three leagues: urban driving, [advanced perception](https://driving-olympics.ai/), and [racing](https://driving-olympics.ai/). The winter champions will be announced during NeurIPS 2021, on December 10, 2021!
[ Urban driving league ](#urban)
[ Advanced perception and racing leagues ](https://driving-olympics.ai/)
## Urban driving league
The urban driving league uses the Duckietown platform and presents several challenges, each of increasing complexity.
The goal in each challenge is to develop a robotic agent for driving Duckiebots “well”. [Baseline implementations](https://docs-old.duckietown.org/daffy/AIDO/out/embodied_strategies.html) are provided to test different approaches. There are no constraints on how your agents are designed.
Each challenge adds a layer of complexity: intersections, other vehicles, pedestrians, etc. You can check out the existing challenges on the Duckietown [challenges server](https://challenges.duckietown.com/v4/humans/challenges).
AI-DO 2021 features four challenges: lane following (`LF`), lane following with intersections (`LFI`), lane following with vehicles (`LFV`) and lane following with vehicles and intersections, multi-body, with full information (`LFVI-multi-full`).
All challenges have a simulation and hardware component (,), except for `LFVI-multi-full`, which is simulation () only.
The **first phase** (until Nov. 7) is a **practice** one. Results do **not count** towards leaderboards.
The **second phase** (Nov. 8-30) is the live **competition** and results count towards official leaderboards.
Selected submissions (that perform well enough in simulation) will be evaluated on hardware in Autolabs. The submissions scoring best in Autolabs will access the finals.
During the **finals** (Dec. 1-8) **one additional submission** is possible for each finalist, per challenge.
Winners (top 3) of the resulting leaderboard will be declared AI-DO 2021 winter champions and celebrated live during NeurIPS 2021. We require champions to submit a short video (2 mins) introducing themselves and describing their submission.
Winners are invited to join (not mandatory) the **NeurIPS event**, on **December 10th, 2021,** **starting at 11.25 GMT** (Zoom link will follow).
##### Overview
🎯**Goal**: develop robotic agents for challenges of increasing complexity🚙**Robot**: [Duckiebot](https://get.duckietown.com/collections/dt-robots/products/duckiebot-db21-m) (`DB21M/J`)👀**Sensors**: camera, wheel encoders
##### Schedule
**Practice**: Nov. 1-7**Competition**: Nov. 8-30**Finals**: Dec. 1 – 8**Winners**: Dec. 10
##### Rules
**Practice**: unlimited non-competing submissions**Competition**: best in sim are evaluated on hardware in Autolabs**Finals**: one additional submission for Autolabs**Winners**: 2 mins video submission description for NeurIPS 2021 event.
## The challenges
### Lane following  
`LF` – The most traditional of AI-DO challenges: have a Duckiebot navigate a road loop without intersection, pedestrians (duckies) nor other vehicles. The objective is to travel the longest path in a given time while staying in the lane, i.e., not committing driving infractions.
Current AI-DO leaderboards: [LF-sim-validation](https://challenges.duckietown.com/v4/humans/challenges/aido-LF-sim-validation/leaderboard), [LF-sim-testing](https://challenges.duckietown.com/v4/humans/challenges/aido-LF-sim-testing/leaderboard).
Previous AI-DO leaderboards: [sim-validation](https://challenges.duckietown.com/v4/humans/challenges/aido5-LF-sim-validation/leaderboard), [sim-testing](https://challenges.duckietown.com/v4/humans/challenges/aido5-LF-sim-testing/leaderboard), [real-validation](https://challenges.duckietown.com/v4/humans/challenges/aido5-LF-real-validation/leaderboard).
[ Get the LF hardware kit ](https://get.duckietown.com/collections/ai-do-kits/products/ai-do-lane-following-lf-challenge-kit)
### Lane following with intersections  
`LFI` – This challenge builds upon `LF` by increasing the complexity of the road network, now featuring 3 and/or 4-way intersections, defined according to the [Duckietown appearance specifications](https://docs.duckietown.com/daffy/opmanual-duckietown/appearance_specifications/index.html). Traffic lights will not be present on the map. The objective is to drive the longest distance while not breaking the rules of the road, now more complex due to the presence of traffic signs.
Current AI-DO leaderboards: [LFI-sim-validation](https://challenges.duckietown.com/v4/humans/challenges/aido-LFI-sim-validation/leaderboard), [LFI-sim-testing](https://challenges.duckietown.com/v4/humans/challenges/aido-LFI-sim-testing/leaderboard).
Previous AI-DO leaderboards: [sim-validation](https://challenges.duckietown.com/v4/humans/challenges/aido5-LFI-sim-validation/leaderboard), [sim-testing.](https://challenges.duckietown.com/v4/humans/challenges/aido5-LFI-sim-testing/leaderboard)
[ Get the LFI hardware kit ](https://get.duckietown.com/collections/starter-kits/products/db-mooc-kit)
### Lane following with vehicles  
`LFV` – In this traditional AI-DO challenge, contestants seek to travel the longest path in a city without intersections nor pedestrians, but with other vehicles on the road. Non-playing vehicles (i.e., not running the user’s submitted agent) can be in the same and/or opposite lanes and have variable speed.
Current AI-DO leaderboards: [LFV-sim-validation](https://challenges.duckietown.com/v4/humans/challenges/aido-LFV-sim-validation/leaderboard), [LFV-sim-testing](https://challenges.duckietown.com/v4/humans/challenges/aido-LFV-sim-testing/leaderboard).
Previous AI-DO leaderboards: (`LFV-multi` variant): [sim-validation](https://challenges.duckietown.com/v4/humans/challenges/aido5-LFV_multi-sim-validation/leaderboard), [sim-testing](https://challenges.duckietown.com/v4/humans/challenges/aido5-LFV_multi-sim-testing/leaderboard), [real-validation](https://challenges.duckietown.com/v4/humans/challenges/aido5-LFV_multi-real-validation/leaderboard).
[ Get the LFV kit ](https://get.duckietown.com/collections/ai-do-kits/products/ai-do-lane-following-with-vehicles-lfv-challenge-kit)
### Lane following with vehicles and intersections (stateful) 
`LFVI-multi-full` – this debuting challenge brings together roads with intersections and other vehicles. The submitted agent is deployed on all Duckiebots on the map (`-multi`), and is provided with full information, i.e., the state of the other vehicles on the map (`-full`). This challenge is in simulation only.
Leaderboards: [LFVI\_multi\_full-sim-validation](https://challenges.duckietown.com/v4/humans/challenges/aido-LFVI_multi_full-sim-validation)
## Getting started
All you need to get started and participate in the AI-DO is a computer, a good internet connection, and the ambition to challenge your skills against the international community!
We provide webinars, operation manuals, and baselines to get started.
May the duck be with you!
[ Participate in AI-DO 2021 ](https://docs-old.duckietown.org/daffy/AIDO/out/quickstart.html)
[ Get AI-DO hardware kit ](https://get.duckietown.com/collections/ai-do-kits)
[ Join our Slack ](https://join.slack.com/t/duckietown/shared_invite/enQtNTU0Njk4NzU2NTY1LWM2YzdlNmJmOTg4MzAyODc2YTI3YTc5MzE2MThkZGUwYTFkZWQ4M2ZlZGU1YTZhYjg5YTgzNDkyMzI2ZjNhZWE)
### Thank you to our generous sponsors!
[  ](https://motional.com/)
[  ](https://www.swissre.com/)
**Categories:** Events, News, Research
**Tags:** AI Driving Olympics, AI-DO, announcement, competition, global, research
---
### [AI Driving Olympics 2021: Urban League Finalists](https://duckietown.com/ai-do-5-leaderboard-update-2-2-2/)
**Published:** December 8, 2021
**Author:** Jacopo Tani
**Content:**
## AI Driving Olympics 2021 - Urban League Finalists
This year’s embodied [urban league challenges](https://www.duckietown.com/archives/82927) were **lane following** (`LF`), **lane following with vehicles** (`LFV`) and **lane following with intersections**, (`LFI`). To account for differences between the real world and simulation, this edition finalists can make one additional submission to the `real` challenges to improve their scores. Finalists are the authors of AI-DO 2021 submissions in the top 5 ranks for each challenge. This year’s finalists are:
### LF
- András Kalapos
- Bence Haromi
- Sampsa Ranta
- ETU-JBR Team
- Giulio Vaccari
### LFV
- Sampsa Ranta
- Adrian Brucker
- Andras Beres
- David Bardos
### LFI
- András Kalapos
- Sampsa Ranta
- Adrian Brucker
- Andras Beres
The deadline for submitting the “final” submissions is **Dec. 9th, 2 pm CET**. All submissions received after this time will count towards the next edition of AI-DO.
Don’t forget to join the [\#aido channel](https://duckietown.slack.com/archives/CAUJLLDGR) on the Duckietown Slack for updates!
[ LF-real Leaderboards ](https://challenges.duckietown.com/v4/humans/challenges/aido-LF-real-validation/leaderboard)
[ LFV-real Leaderboards ](https://challenges.duckietown.com/v4/humans/challenges/aido-LFV-real-validation/leaderboard)
[ LFI-real Leaderboards ](https://challenges.duckietown.com/v4/humans/challenges/aido-LFI-real-validation)
Congratulations to all the participants, and best of luck to the finalists!
**Categories:** dep-News, Events, Research
**Tags:** AI Driving Olympics, aido2021, finals
---
### [Learning Autonomy with Vincenzo Polizzi](https://duckietown.com/learning-autonomy-in-practice-with-vincenzo-polizzi/)
**Published:** March 16, 2022
**Author:** Ivano Marocchi
**Content:**
**ETHZ, Zurich, March 11, 2022**: How Vincenzo discovered his true professional passion as a student using Duckietown.
## Learning Autonomy in practice with Vincenzo Polizzi
Vincenzo Polizzi studied robotics, systems and control at the Swiss Federal institute of Technology (ETH Zurich). Vincenzo shares below his experience with Duckietown. Starting off as a student, becoming a Teaching Assistant and onto how he uses Duckietown to power his own research as he moves from academia to industry.
##### Quick links
- [ ETH Zurich ](https://ethz.ch/en.html)
- [ DuckVision Project repo ](https://github.com/viciopoli01/DuckVision)
- [ Parking Project repo ](https://github.com/duckietown-ethz/proj-parking)
**Could you tell us something about yourself?**
I’m Vincenzo Polizzi , I studied automation engineering at the Politecnico di Milano, and I currently study robotics, systems and control at the Swiss Federal Institute of Technology (ETH Zurich).
**You use Duckietown. Could you tell us when you first came into contact with the project, and what attracted you to Duckietown?**
Sure! I learned about Duckietown my first year during a master’s program at ETH, where there was a course called: “Autonomous mobility on demand, from car to fleet” where I saw these cars, these robots. And I asked myself “what is this thing?”. It seemed very interesting. The first thing that struck me was that it did not look theoretical, but clearly practical.
> "It captures you with simplicity and then you stay for the complexity."
>
> Vincenzo Polizzi
**So the idea of a practical aspect interested you?**
Yes, during the presentation, it was clear that the course was based on projects the student had to carry out, where one could practice what they had learned theoretically in other classes.
I come from a scientific high school, and I studied automation engineering in Milan. In both my study experiences, I was used to learning concepts theoretically. For example, in the control system for a plant you design on paper in university, you don’t really face the complexity of implementing it on a real object.
I have to say that I have always been very passionate about robotics and informatics. In fact, even in high school, I was building these little robots,I participated in the robotics competition Rome Cup held by Fondazione Mondo Digitale, and there were these robots that were similar in shape to those of Duckietown, but where the scientific content was completely different. So in Duckietown, I saw something similar to what I was doing in my free time. I wanted to see exactly how it was inside, and there I discovered a whole other world that is obviously much more scientific than what a normal high school student could imagine by themself. However, initially I was curious to see a course where one can practice all the knowledge they have gradually acquired. It is not just about writing an equation and finding a solution but making things work.
**What is your relationship with Duckietown, how long have you been using it? How do you interact with the Duckietown ecosystem? How do you use it, what do you do with it?**
These are interesting questions because I started as a student and then managed to see what’s behind Duckietown. I was attending the course Duckietown held at ETH in 2019. The class was limited to 30 students, I was really excited to be part of it. I met many excellent students there, some of whom I am still in touch with today.
When I started the course, I immediately told myself, “Duckietown is a great thing. If all universities used Duckietown, this would be a better world.” I liked the class a lot, then I also had the opportunity of being a TA. The TAship was an important step because I learned more than during the course. One thing is to live the experience as a student who has to take exams, complete various projects, etc. You need a deeper understanding to organize an activity. You have to take care of all the details and foresee the parts of the exercise that can be harder or simpler for the students. This experience helped me a lot. For example, I did an internship in Zurich where we had to develop a software infrastructure for a drone, and I found myself thinking, “wow this can be done with Duckietown, we can use the same technologies.” I noticed that even in the industry, often we see the use of the same technologies and tools that you can learn about thanks to Duckietown. Of course, maybe a company has its own customized tools, probably well optimized for its products. Perhaps it uses some other specific tool but let’s say you already know more or less what these tools are about. You know because in Duckietown, you have already seen how a robotics system should work and the pieces it is composed of. Duckietown has given me a huge boost with the internship and my Master’s thesis at NASA JPL. Consider that my thesis was on a system of multi drones, so I used, for example, Docker as a tool to simulate the different agents. With Duckietown, I acquired technical knowledge that I used in many other projects, including work.
**Do you still use it today?**
The last project I did with Duckietown is DuckVision. I know we could have thought of a better name. With one of my Duckietowner friends, Trevor Phillips, we enhanced the Duckiebot perception pipeline with another camera, a stereocamera made by Luxonis and Open CV called OAKD (OpenCV AI Kit with depth). This sensor is not just a simple camera, but it also mounts a VPU, Visual Processing Unit. Namely, it can analyze and make inferences on the images that the camera acquires onboard. It can perform object detection and tracking, gesture recognition, semantic segmentation, etc. There are plenty of models freely available online that can run on the OAK-D. We have integrated this sensor in the Duckietown ecosystem, using a similar approach used in the MOOC “Self-Driving Cars with Duckietown”, we created a small series of tutorials where you can just plug the camera on the robot, run our Docker container and have fun! With this project, we passed the first phase of the OpenCV AI Competition 2021. The idea behind the project was to increase the Duckiebot understanding of the environment, by using the depth information, the robot can have a better representation of its surroundings and so, for example, a better knowledge of its position. Also, in our opinion, the OAK-D in Duckietown can boost the research in autonomous vehicles and perception.
I would like to add something about the use of Duckietown, I have seen this project both as a student and from behind the scenes and I really understood that by using this platform you really learn a lot of things that are useful not only in the academic field but can also be very useful in the working environment with the practical knowledge that is often difficult to acquire during school. And in this regard I thought then given my history, I am Sicilian but I studied in Milan and then I went to Zurich, I asked myself what can I bring as a contribution of my travels, so I thought about using Duckietown in some universities here in Sicily in the universities of Palermo and Messina. And also, at the Polytechnic of Milan, for example, they have already begun to use it and have participated in the AIDO and have also placed well, they ended up among the finalists, so there is a lot of interest in this project.
**Did you receive a positive response every time you proposed Duckietown?**
Yes, and then there is a huge enthusiasm on the part of the students. I spoke with student associations first, then with the professors etc. but when the students see Duckietown for the first time, they are always really enthusiastic about using it.
> "There is something that captures you in some way, and then just opens up a world when you start to actually see how all the systems are implemented. This is the nice thing in my opinion, you can decide the level of complexity you want to achieve."
>
> Vincenzo Polizzi
**The duck was a great idea!**
Absolutely right! The duck was a great idea, yes. I like contrasts, you see a super simple friendly thing that hides a state-of-the-art robotics platform. Even in the students I saw this reaction, because the duck is the first thing you see, it looks like a game, something to play with, this is the first impact, then when you start you get curious. It captures you with simplicity and then you stay for the complexity.
**Would you suggest Duckietown to friends and colleagues?**
Sure! There is something that captures you and opens up a world when you start to see how all the systems are implemented. This is the nice thing in my opinion, you can decide the level of complexity you want to achieve. It’s a platform that looks like something to play with, a game or something, but in reality there is a huge potential, in terms of knowledge that everyone can acquire, it’s something that you can not easily find elsewhere. I also think it offers great support, such as educational material, exercises that are of high quality. You can learn a lot of different aspects of robotics, in my opinion. You can do control, you can do the machine learning part, perception . There’s really a world to explore. You can see everything there is about robotics. But you can also just focus on one aspect that maybe you’re more passionate about. So yes, I would recommend it because you can learn a lot, and as a student myself I would recommend it to my fellow colleagues.
### Learn more about Duckietown
The Duckietown platform offers robotics and AI learning experiences.
Duckietown is modular, customizable and state-of-the-art. It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Find out more use cases ](https://www.duckietown.com/news/people-of-duckietown)
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** Blog, News, People
---
### [When rubber duckies meet the road: an interview with Prof. Liam Paull](https://duckietown.com/when-rubber-duckies-meet-the-road-an-interview-with-prof-liam-paull/)
**Published:** May 10, 2022
**Author:** Ivano Marocchi
**Content:**
**UdM, Montréal, May 5, 2022**: Liam Paull, professor at the University of Montreal and one of Duckietown’s founders, talks about his role and experiences with Duckietown.
## When rubber duckies meet the road: an interview with Prof. Liam Paull
Liam Paull, professor at the University of Montreal in Quebec, and one of the very founders of Duckietown, shares below his unique perspective about Duckietown’s journey and its origin.
##### Quick links
- [ Université de Montréal: DIRO ](https://diro.umontreal.ca/accueil/)
- [ AI Driving Olympics (AI-DO) ](https://www.duckietown.com/research/AI-Driving-olympics)
- [ Duckietown Massive Online Open Course ](https://www.duckietown.com/mooc)
**Good morning, Liam.**
Hello.
**Thank you very much for accepting to have this little chat.
Could you tell us something about you?**
Sure. So my name is Liam Paull. I’m a professor at the University of Montreal in Quebec, Canada. I teach in the computer science Department, And I do research on robotics.
**Ok and when was the first time you “came across” Duckietown?**
Well, I’m actually one of the creators of Duckietown, So I didn’t come across it as much! The origin story of Duckietown is kind of interesting, But I probably forgot some of the details. It must have been about 2015. And myself, Andrea Censi, and a few others were interested to get more teaching experience. We were all postdocs or research scientists at MIT at the time. I guess we started brainstorming ideas, and then roughly around that time, I switched positions at MIT. I was previously a postdoc in John Leonard’s group working on marine robotics, and then I switched to become part of Danielle Lerous lab and lead an autonomous driving project. And so somehow the stars just aligned. That the right topic for this class that we would teach would be autonomous driving. Yeah, the Ducky thing is kind of a separate thing. Actually, Andrea had started this other thing that was making videos for people to publicize their work at a top robotics conference Called the international conference robotics automation, and somehow had the idea that every single video that was submitted should have a rubber Ducky in it. And this was for scale or something.
There was some kind of reason behind it I sort of forget. But anyway, so the branding kind of caught fire.
When we were building the class, we agreed the one constraint was that there should be duckies involved somehow, and the rest is kind of history!
**What’s your relationship with Duckietown today? Like, do you use it in particular for some activities, your daily work or some project?**Yeah, for sure. I guess I use it in a number of ways. Maybe the first way is that I teach a class every fall called autonomous vehicles, Where the Duckietown platform is the platform that we use for the experiments and labs in the class. So just like the original class, Every student gets a robot that they assemble, and then we learn about computer vision and autonomous driving and all the good stuff related to robotics. But I also use the platform for some amount of research. Also in my group, I believe that there’s a lot of interesting research directions that come from a kind of standardized, small scale, accessible autonomous driving platform like this. Recently, most of the work that we’ve been doing in terms of research has been about training agents in simulation and then deploying them in the real world. So this isa nice setup for that because we have a simulator that’s very easy, fast and lightweight to train in, and then we have the environment that’s also really accessible. So, yeah, so we’ve been doing some research on that front.
**So would you recommend Duckieown to colleagues or students of yours? And if yes, why.**Of course. I think that’s what’s nice. Going back to the original motivation behind building Duckietowng and some of the tenets: thee guiding principle for us was this idea that to learn robotics, you have to get your hands on a robot. And we are also very adamant that it should be that every student should have their own robot. With teams of robots or going into the lab and only being able to use the robot at certain hours. It’s something funny.
You don’t develop the same kind of personal relationship. It sounds weird, but it’s true. Like when you have your own thing that you’re working with every day, you have some kind of bond with that thing, and you develop some kind of love or hate or whatever the case may be depending on how things are going on that particular day. So I think that with this set up, we have a platform where we’ve scaled things down and made things cost effective, to be able to do that. We built an engaging, experimental platform where it’s totally, I think, reasonable for most University budgets to be able to get their hands on the hardware.
> "I believe that there's a lot of interesting research directions that come from a standardized, small scale, accessible autonomous driving platform like Duckietown."
>
> Liam Paull
The other big piece is the actual teaching materials that we’ve developed. And I think that we have some good stuff. It could be better. Some stuff could be better, but that’s where we also need the community to come in. I mean, if we have this standardized platform and lots of people start using it and building educational experiences around the platform, then the entire thing just starts to get better and better for everybody. And it just grows into a very nice thing where you can also pick and choose the pieces that you want to include for your particular class, and you can customize the experience of what your class is going to look like using all of the resources that are out there. Also, the other part that I’ve really tried to cultivate, this is sort of a new thing. When we ran the first class at MIT, it was really an isolated thing. But in the subsequent iterations of the class, like myself and others have been in different places around the world, whether it’s Matt Walter at TTIC or Jacopo and Andrea at ETH. So we tried to turn the class into this kind of global experience, where you feel like you’re part of something that’s bigger than just the class that you’re taking at the specific University. And I think students really like that. We’ve experimented with different models where people do projects with other students from other universities or even just feeling part of the global community. I think it’s a very fun and engaging. Students are so connected these days. They’re so plugged in. They like this aspect of feeling like there’s a bit more of a broad social aspect, too. So I think these are some of the elements that this platform project experience brings to the table that I don’t see replicated and too many other setups.
**Anything else you would like to add about Duckietown and it’s uses?**
I didn’t mention specifically about the MOOC. One of the core missions of this project from the onset has been that it’s accessible. Both in terms of hardware but also in terms of software. And part of what that means to us Is that no matter where you are, no matter who you are, you should be able to get the hardware and you should be able to use the educational resources to learn. And part of the motivation for that Was that we saw that while we were at MIT. When you’re at a place like MIT you are extremely privileged and if you come from a background of less privilege, you see the discrepancy. In some sense, it’s palpable. Part of that, I guess, was that we don’t even necessarily want it to be a prerequisite that students should be enrolled in universities in order to be able to address the platform. So we built this massive online open source course through edx, which is also an open source provider Where people can, regardless of their background or regardless of their situation, they can sign up for this thing, and it’s a creative set of materials that also have exercises to interact with the robot that anybody can do, Regardless of whether they’re at a University or not.
I think this is the next step for us in making the platform accessible to all, and we’re going to continue to run iterations of this thing. But I also think that this is an exciting objective that very much fits in the mission of what we’re trying to do with this project.
**This was great thank you for your time!**
Awesome. Great. Thank you for your time. Bye.
### Learn more about Duckietown
The Duckietown platform offers robotics and AI learning experiences.
Duckietown is modular, customizable and state-of-the-art. It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Find out more use cases ](https://www.duckietown.com/news/people-of-duckietown)
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** Blog, editor-choiche, News, People
---
### [Join the new “Self-Driving Cars with Duckietown” MOOC](https://duckietown.com/self-driving-cars-with-duckietown-mooc-starting-soon-2/)
**Published:** November 11, 2022
**Author:** Ivano Marocchi
**Content:**
## Join the self-driving cars with Duckietown MOOC user-paced edition
Over 7200 learners engaged in a robotics and AI learning adventure with **“Self-Driving Cars with Duckietown”**, the first massive online open course (MOOC) on robot autonomy with hardware, hosted on the edX platform.
Kicking off on **November 29th**, this new edition is a **user-paced** course with rich and engaging modules offering a **grand tour of real-world robotics**, from computer vision to perception, planning, modeling, control, and machine learning, released all at once!
With **simulation and real-world** learning activities, learners can touch with hand the emergence of autonomy in their robotic agents with approaches of increasing complexity, **from Brateinberg vehicles to deep learning** applications.
We are thrilled to welcome you to the start of the second edition of Self-Driving Cars with Duckietown.
This is a new learning experience in many different ways, for both you and us. While the course is self-paced, the instructors and staff, as well as your peer learners and the community of those that came before you are standing behind, ready to intervene and support your efforts at any time.
Learn autonomy hands-on by making real robots take their own decisions and accomplish broadly defined tasks. Step by step from the theory, to the implementation, to the deployment in simulation as well as on Duckiebots.
Leverage the power of the **NVIDIA Jetson Nano-powered Duckiebot** to see your algorithms come to life!
[  ](https://get.duckietown.com/)
[ Enroll for free on edX ](https://www.edx.org/course/self-driving-cars-with-duckietown)
### MOOC Factsheet
- Name: Self-driving cars with Duckietown
- [ Platform: edX ](https://www.edx.org/course/self-driving-cars-with-duckietown)
- Cost: free to attend
- Instructors: Swiss Federal Institute of Technology in Zurich (ETHZ), Université de Montréal (UdM), Toyota Technological Institute at Chicago (TTIC)
### Prerequisites
- Basic Linux, Python, Git
- Elements of linear algebra, probability, calculus
- Elements of kinematics, dynamics
- Computer with native Ubuntu installation
- Broadband internet connection
### What you will learn
- Computer Vision
- Robot operations
- Object Detection
- Onboard localization
- Robot Control
- Planning
- Reinforcement Learning
[ Get the hardware ](https://get.duckietown.com/products/db-mooc-kit?variant=41597038395567)
[ Subscribe for updates ](/contact)
Designed for university-level students and professionals, this course is brought to you by the **Swiss Federal Institute of Technology in Zurich (ETHZ)**, in collaboration with the **University of Montreal**, the **Duckietown Foundation**, and the **Toyota Technological Institute at Chicago**.
[  ](https://ethz.ch/en.html)
**Learning autonomy** requires a **fundamentally different approach** when compared to other computer science and engineering disciplines. Autonomy is inherently **multi-disciplinary**, and mastering it requires expertise in domains ranging from fundamental mathematics to practical machine-learning skills.
[  ](https://www.umontreal.ca/en/)
[  ](https://www.ttic.edu)
[ Watch the videos ](https://vimeo.com/showcase/8807247)
The Duckietown robotic ecosystem was created at the **MIT** Computer Science and Artificial Intelligence Laboratory (CSAIL) in 2016 and is now used in over 175 universities worldwide.
“The Duckietown educational platform provides a hands-on, scaled-down, accessible version of real-world autonomous systems.” said **Emilio Frazzoli**, **Professor of Dynamic Systems and Control, ETH Zurich**, “Integrating NVIDIA’s Jetson Nano power in Duckietown enables unprecedented access to state-of-the-art compute solutions for learning autonomy.”
"I’m thrilled that ETH, with UMontreal, the Duckietown Foundation, and the Toyota Technological Institute in Chicago, are collaborating to bring this course in self-driving cars and robotics to the 35 million learners on edX. This emerging technology has the potential to completely change the way we live and travel, and the course provides a unique opportunity to get in on the ground floor of understanding and using the technology powering autonomous vehicles."
Anant AgarwalFounder and CEO of edX, Professor at the Massachussetts Institute of Technology (MIT)
"The new NVIDIA Jetson Nano is the ultimate starter AI computer for educators and students to teach and learn AI at an incredibly affordable price. Duckietown and its EdX MOOC are leveraging Jetson to take hands-on experimentation and understanding of AI and autonomous machines to the next level."
Deepu TallaVice President and General Manager of Edge Computing at NVIDIA
"The Duckietown educational platform provides a hands-on, scaled down, accessible version of real world autonomous systems. Integrating NVIDIA’s Jetson Nano power in Duckietown enables unprecedented access to state-of-the-art compute solutions for learning autonomy."
Emilio FrazzoliProfessor at the Swiss Federal Institute of Technology in Zurich (ETHZ)
**Enroll now** and don’t miss the chance to join in the first vehicle autonomy MOOC with hands-on learning!
[ Enroll for free on edX ](https://www.edx.org/course/self-driving-cars-with-duckietown)
**Categories:** Classes, dep-News, editor-choiche
**Tags:** DB21, MOOC, teaching
---
### [Introduction to robotics at the University of Massachusetts Lowell using Duckietown](https://duckietown.com/introduction-to-robotics-at-the-university-of-massachusetts-lowell-using-duckietown/)
**Published:** December 19, 2022
**Author:** Ivano Marocchi
**Content:**
**University of Massachusetts, Lowell, December 20, 2022**: Paul Robinette, Assistant Professor at the University of Massachusetts Lowell (UML), shares with us his Duckietown teaching experience.
## Introduction to robotics at the University of Massachusetts Lowell using Duckietown
Paul Robinette is an Assistant Professor in the Department of Electrical and Computer Engineering at the University of Massachusetts Lowell. He shares his experience, and that of his students, using Duckietown for teaching throughout the years. His “Fundamentals of Robotics” (EECE 4560/5560) course with Duckietown platform has been repeating since 2019.
##### Quick links
- [ University of Massachusetts Lowell (UML) ](https://www.uml.edu/)
- [ Human-robot teaming experiments in the marine domain ](https://oceanai.mit.edu/aquaticus)
- [ (EECE 4560/5560) Fundamental of Robotics 2022 ](https://sites.uml.edu/paul-robinette/teaching/eece-5560-fall-2022/)
**Thank you for finding the time to talk with us! Could you introduce yourself?**
My name is Paul Robinette \[[Linkedin](https://www.linkedin.com/in/paul-robinette-24857b/)\] and I’m a Professor of computer engineering at the University of Massachusetts Lowell \[[website](https://sites.uml.edu/paul-robinette/)\].
**When was your first experience with Duckietown?**
My first experience with Duckietown would have been when I worked at MIT as a research scientist, just after Duckietown was run. I didn’t have a chance to see it live there, but I did talk with several of the postdocs who worked on it as it ran. I also saw it live for the first time at ICRA 2019.
**Do you use Duckietown or did you use Duckietown in the past for some specific project or activity?**
Sure! For the last three years, I’ve been using Duckietown robots in my class every semester. Primarily I use the Duckiebots to teach ROS and basic robot skills through the Duckietown system and infrastructure. I leverage the development infrastructure heavily and some of the course materials as well.
**That sounds great! Can you tell us more about your ongoing class?**
The class I teach every semester so far is called Fundamentals of Robotics \[[2022 class page](https://sites.uml.edu/paul-robinette/teaching/eece-5560-fall-2022/)\], and we go over the basics of robotics, starting with multi-agent processing or multi-process systems, like most robots are these days, some basic networking problems, etc. The Duckiebots are perfect for that because they have Docker containers on board which have multiple different networks running. They have to work with the computer system, so it’s always at least interfacing with the laptop. The robots can be used with a laptop, with a router, you can have multiple robots out at once, and they give the students a really good sense of what moving real robots around feels like. We have students start by implementing some open loop control systems, then have them design their own lane detector, similar to the Duckietown \[perception\] demo, and then have them design their own lane controller again, similar to the Duckietown \[lane following\] demo.
**Are your students appreciating using Duckietown? Would you consider it a success?**
Yes, especially the newer version. The DB21s are great robots for using their class applications and infrastructure. The software infrastructure has made it pretty easy to set up our own Git repositories for the robots and be able to run them. In this way, students can run them at home or on campus.
**Would you suggest Duckietown to your students or colleagues?**
Yes, I’d suggest Duckietown, especially if people want to run an introductory robotics class and have every student purchase their own robot, or have the University provide the robots for all. Duckietown is much more affordable than any other robot system that could be used for this same purpose.
> "In my class we go over the basics of robotics, starting with multi-agent processing or multi-process systems, like most robots are these days, and the Duckiebots are perfect for that"
>
> Prof. Paul Robinette
**It is great to hear Duckietown addresses your needs so well. What would you say is the advantage that Duckietown has when compared to other systems?**
I’d say the expense is probably the biggest advantage right now. It’s a nice platform and very capable for what we wanted to do. At this point, the fact that it’s affordable for students to purchase on their own or for us to purchase a bunch of them is definitely the biggest advantage for us. You guys also have a really quick response time if we have any problems. It’s nice to be able to talk directly with the development team and work with them to set up the systems so that I can run them in my class as I need to.
**Thank you very much for your time!**
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences.
It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
[ Find out more use cases ](https://www.duckietown.com/news/people-of-duckietown)
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** Blog, dep-News, News, People
---
### [Shima Akbari graduates with Duckietown at La Sapienza University in Rome](https://duckietown.com/shima-akbari-graduates-with-duckietown-at-la-sapienza-university-in-rome/)
**Published:** April 23, 2023
**Author:** Ivano Marocchi
**Content:**
**“La Sapienza” University of Rome, April 28, 2023**: Shima Akbari, a Ph.D. student at the Italian National Program in Autonomous Systems, shares her experience working with Duckietown for her master’s thesis on lane following control for mobile robots.
##### Quick links
- [ Sapienza University of Rome ](https://www.uniroma1.it/en/pagina-strutturale/home)
- [ Italian National Program in Autonomous Systems ](http://dausy.poliba.it/phd/)
- [ Shima Akbari LinkedIn ](https://www.linkedin.com/in/shima-a-3115391a0/)
## Shima's work on control strategies for lane following
**Hi, thank you for joining us today. Could you introduce yourself?**
Certainly. My name is Shima Akbari, and I have a degree in Control Engineering from “La Sapienza” University of Rome. Currently, I am a Ph.D. student in the Italian National Program in Autonomous Systems.
**You recently obtained your master’s degree, congratulations! You worked on lane following control approaches for mobile robots using Duckietown. How did you discover Duckietown?**
I first learned about Duckietown while working on my master’s thesis. I have always been passionate about control and autonomy, particularly in mobile robots. However, I didn’t want to limit my project to theoretical calculations or computer simulations. I wanted to have a practical component to my work. When I shared this with my supervisor, Professor Oriolo, he introduced me to Duckietown and suggested that I conduct my experiments in this environment. So I implemented the lane following control in the Duckietown environment as part of my master’s thesis, using both the Duckietown simulator called Duckietown Gym, and the Duckiebots, which are the robots used in Duckietown. I thoroughly enjoyed every minute of conducting my tests.
**It’s great to hear you enjoyed working with Duckiebots! Tell us a little more about your project and what was your experience like.**
My thesis focus was on computer vision based control. I used OpenCV, the well known computer vision library, and the camera mounted on the Duckiebots to extract lane lines from the streets in Duckietown. Based on information extracted from these features, I implemented control laws that enabled the Duckiebot to drive on the streets inside lanes.
To familiarize myself with the platform, I started by taking the Duckietown massive open online course on edX and completed the assignments and homework on my own. One of the modules was about implementing a PID controller for lateral position and another on steering rate control. I enjoyed the Braitenberg vehicles activity too, but my favorite project was on obstacle avoidance, obstacle detection, and computer vision.
> "In engineering, true learning comes from practical implementation, and Duckietown offers that opportunity effectively."
>
> Shima Akbari
**You are studying in a field that is statistically dominated by male presence. What are your thoughts on this?**
It’s indeed the case. According to recent statistics, only 16% of women are in engineering compared to 84% of men. While I acknowledge this disparity, I believe that women are just as capable as men in engineering or any other field. Moreover, I think that the situation is improving over time. If we look at statistics from 10 to 20 years ago, the percentage of women in engineering was even lower.
**What would you say to a young woman who wants to study engineering and may be discouraged by the statistics?**
I would tell her that statistics and other people’s opinions should not deter her from pursuing her interest in engineering or any other subject. She should follow her dreams and not be discouraged by external factors.
**Thank you for this thought, we hope this interview will help it reach as many women thinking about pursuing engineering careers out there as possible.**
Absolutely. I would recommend Duckietown to anyone interested in learning about autonomous systems, regardless of their background or gender. It provides an excellent opportunity to learn about autonomy and control in a practical and user-friendly way. In engineering, true learning comes from practical implementation, and Duckietown offers that opportunity effectively.
**What would you consider to be the unique value or appeal of Duckietown? What makes it special?**
I would say that the simplicity of Duckietown is its most appealing aspect. The robots are designed to be simple and easy to use, and working with them is a lot of fun. Additionally, Duckietown has excellent support, with comprehensive documentation and manuals that are written in a detailed, step-by-step manner. Even if you don’t have a strong background in tools like Linux or Docker, you can still make progress by reading and following the documentation
> "Duckietown is simple and has excellent support, with comprehensive documentation and manuals that are written in a detailed, step-by-step manner. Even if you don't have a strong background in concepts like Linux or Docker, you can still make progress by reading and following the documentation."
>
> Shima Akbari
**Thank you very much for taking the time to share your experience with us, we really appreciate it. Is there anything else you would like to add?**
I’d just like to express how amazing it was for me to be introduced to and work with Duckietown. I would highly recommend it to others as well.
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
[ Find out more use cases ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** Blog, News, People
---
### [Robotics education and research at IIT Jodhpur: an interview with Prof. Debasis Das](https://duckietown.com/people-of-duckietown-iit-jodhpur-debasis-das/)
**Published:** June 7, 2023
**Author:** Duckietown Admin
**Content:**
**IIT Jodhpur, June 03, 2023**: Prof. Debasis Das of IIT Jodhpur, Rajasthan, India, tells us about his experience in bringing Duckietown to his lab to support the teaching, research, and outreach efforts of his team in the fields of autonomous vehicles, robotics, computer vision, machine learning.
##### Quick links
- [ IIT Jodhpur ](https://www.iitj.ac.in/)
- [ VANET@IITJ Lab ](https://vanets-iitj.github.io/)
- [ Prof. Debasis Das ](https://www.linkedin.com/in/dr-debasis-das-42839a26/?originalSubdomain=in)
## Robotics education and research at IIT Jodhpur: an interview with Prof. Debasis Das
**Hello Professor Das, thank you for your time in taking our questions. How did you learn about Duckietown?**
I learned about the Duckietown project through a variety of sources, including online platforms like social media, discussion forums, and academic publications, where many researchers and robotics enthusiasts have shared information related to the project. Additionally, some of my collaborators have also mentioned the project and its educational and research goals. Through these sources, I gained an understanding of what the Duckietown project is and the impact it has had in the field of robotics education and research.
**This is great to hear! Starting from the education aspects, what classes are you teaching at IIT Jodhpur using Duckietown?**
Duckietown is being used for a variety of classes and educational programs, primarily in the fields of computer science and engineering at IIT Jodhpur. Most recently we have utilized the platform to teach the following classes:
- Vehicular Ad Hoc Networks: This class focuses on the design, development, and testing of autonomous vehicles using the Duckietown platform to simulate real-world scenarios.
- Robotics and Mobility: This class teaches students the principles of mobility and the role of machine learning for decision-making, using the Duckietown platform for practical exercises.
**What has your and your students’ experience been in these courses?**
We have found that hands-on learning experiences, such as those provided by Duckietown, are more effective in increasing students’ knowledge and capacity to recall topics than standard lecture-based teaching approaches.
Furthermore, the dynamic and fascinating elements unique to Duckietown increase students’ passion for and interest in their assignments.
We have received large positive feedback for Duckietown from students and researchers who have used it at IIT Jodhpur, with many appreciating its entertaining and challenging nature. The platform’s scalability and ease of use across a wide range of disciplines and programs are commendable. Students also appreciate the opportunity to learn programming and problem-solving with real-world robotics difficulties.
Though there may be some disparities in how each student utilizes the platform and what they gain in terms of education and enjoyment, the data thus far suggests that Duckietown can be a helpful and fun resource for students working in robotics and related fields.
> "The Duckietown platform has been a valuable resource in supporting our research activities. We have used it as a tool for engaging with the broader community and promoting interest in science, technology, engineering, and STEM fields, including hosting workshops, competitions, and other events that showcase the capabilities of autonomous vehicles and provide opportunities for hands-on learning and exploration."
>
> Prof. Debasis Das
**Are you using Duckietown to support your research activities too?**
Yes, the Duckietown platform has been a valuable resource in supporting our research activities. We have used the platform to test and evaluate novel algorithms and methods for autonomous vehicle design and control, and to investigate issues such as computer vision, machine learning, and control systems.
Are there other ways Duckietown has helped you conduct teaching, research, and outreach efforts?
We have used Duckietown as a tool for engaging with the broader community and promoting interest in science, technology, engineering, and mathematics (STEM) fields. We have hosted workshops, competitions, and other events that showcase the capabilities of autonomous vehicles and provide opportunities for hands-on learning and exploration.
**Would you suggest Duckietown to your colleagues?**
Yes, I would definitely recommend Duckietown to my colleagues who are interested in teaching and researching areas such as autonomous vehicles, robotics, computer vision, machine learning, and control systems.
Duckietown provides a realistic and scalable environment for testing and assessing novel algorithms and methods for autonomous vehicle design and control, as well as a platform for engaging students and the general public in learning and discovery in these domains.
> "We have received large positive feedback for Duckietown from students and researchers who have used it at IIT Jodhpur, with many appreciating its entertaining and challenging nature. The platform's scalability and ease of use across a wide range of disciplines and programs are commendable."
>
> Debasis Das
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
[ Find out more use cases ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** Blog, News, People
---
### [Duckietown to join the UK Permanent Science Museum Collection!](https://duckietown.com/duckietown-joins-science-museum-collection/)
**Published:** October 7, 2023
**Author:** Duckietown Admin
**Excerpt:** Duckietown objects joined the Science Museum Group permanent Collection: the UK's national archive of science, technology, engineering, and medicine.
**Content:**
**Boston, October 2023**: several Duckietown objects officially joined the Science Museum Group permanent Collection: the United Kingdom’s national archive of science, technology, engineering, and medicine!
##### Quick links
- [ Permanent Collection ](https://collection.sciencemuseumgroup.org.uk/)
- [ Driverless: Who is in control? ](https://www.sciencemuseumgroup.org.uk/our-services/partner-with-us/touring-exhibitions/driverless-who-is-in-control/)
## Duckietown joins UK's Science Museum Group Collection
Duckietown is a trailblazer in providing accessible solutions for teaching and learning state-of-the-art autonomy, helping in the dissemination of the science and technology of modern robotics.
In 2019, the Science Museum of London picked up on the project and included Duckietown items: a Duckiebot (DB19), a segment of Duckietown, and a Duckietown traffic light, in their “[Driverless: Who is in control?](https://www.sciencemuseumgroup.org.uk/our-services/partner-with-us/touring-exhibitions/driverless-who-is-in-control/ "UK Science Museum Group Collection Driverless Exhibition")” exhibition.
Following the success of this exhibition, the Board of Directors of the Science Museum started considering including the DUckietown items in the permanent collection of their institution.
We are proud to announce that, in August 2023, these Duckietown items have officially joined the Science Museum Group Collection – the UK’s national collection of science, technology, engineering and medicine.
### Duckietown in the history books
Joining the permanent collection will:
“*ensure that the items – as rare and representative objects – are acquired, conserved, preserved and stored in order that they may be accessible to current and future generations for interpretation, loans to other institutions, research, education, and sometimes display in temporary or permanent exhibitions.*”
[ ](https://duckietown.com/wp-content/uploads/2023/10/E2019.0205-5.jpg) [ ](https://duckietown.com/wp-content/uploads/2023/10/E2019.0205-4.jpg) [ ](https://duckietown.com/wp-content/uploads/2023/10/E2019.0205-9.jpg) [ ](https://duckietown.com/wp-content/uploads/2023/10/E2019.0205-10.jpg) [ ](https://duckietown.com/wp-content/uploads/2023/10/E2019.0205-8.jpg) [ ](https://duckietown.com/wp-content/uploads/2023/10/E2019.0205-3.jpg) [ ](https://duckietown.com/wp-content/uploads/2023/10/E2019.0205.jpg) [ ](https://duckietown.com/wp-content/uploads/2023/10/E2019.0205-7.jpg) [ ](https://duckietown.com/wp-content/uploads/2023/10/E2019.0205-6.jpg) [ ](https://duckietown.com/wp-content/uploads/2023/10/E2019.0205-11.jpg) [ ](https://duckietown.com/wp-content/uploads/2023/10/E2019.0205-2.jpg)
here are some details:
2023-216
E2019.0205.1
Duckiebot – small autonomous robot from Duckietown Project, 2018-2019
2023-222
E2019.0205.2
Traffic light kit from Duckietown Project, 2018-2019
2023-502
E2019.0205.3
City expansion pack from Duckietown Project, 2018-2019
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
[ Request a quote for your class ](https://contact.duckietown.com/request-a-quote)
**Categories:** Blog, editor-choiche, Media Coverage, News
**Tags:** duckiebot, duckietown, robotics, science museum
---
### [Practical and playful: Duckietown talks with Prof. Donoso](https://duckietown.com/interview-prof-donoso/)
**Published:** October 11, 2023
**Author:** Duckietown Admin
**Excerpt:** Félix Donoso, Associate Professor at Duoc UC uses Duckietown to teach his students about the science and technology of autonomy in a practical and playful way.
**Content:**
Félix Donoso, Associate Professor at Duoc UC uses Duckietown to teach his students about the science and technology of autonomy in a practical and playful way.
**Santiago de Chile, 17 October 2023:** Learn how Félix Donoso H., Associate Professor at the School of Computer Science and Telecommunications at the Duoc UC Professional Institute, utilized the Duckietown platform to engage with students and teach them complex concepts in a practical and playful way.
##### Quick links
- [ Prof. Donoso ](https://www.linkedin.com/in/felixdonosoh/?originalSubdomain=cl)
- [ Duoc UC ](https://www.duoc.cl/)
- [ Duckietown at Duoc ](https://www.duoc.cl/?noticia_post_type=duckietown-el-proyecto-de-duoc-uc-que-fomenta-el-aprendizaje-con-inteligencia-artificial)
## Practical and playful, an interview with Prof. Donoso
###### Hi! Could you introduce yourself?
My name is Félix Donoso H. I hold a Master’s degree in Education, specializing in Teaching for Higher Education. I am also an Engineer in Connectivity and Networks and have a Diploma in Applied Research and Innovation. I have been working in higher education for 8 years. Currently, I am an Associate Professor at the School of Computer Science and Telecommunications at the Duoc UC Professional Institute. I am involved in various projects at the Center for Innovation and Technology Transfer.
###### We read on the DUOC website about your Duckietown class, could you tell us a little more about it?
At the Center for Innovation and Technology Transfer of the School of Computer Science and Telecommunications at DuocUC, we have recently implemented Duckietown to offer our students hands-on experience in robotics and autonomous vehicles. The workshops are designed to teach fundamental concepts of control, computer vision, and machine learning, using the Duckietown platform to provide a hands-on and cutting-edge yet accessible experience.
###### What is the pedagogical focus of your learning activities?
The pedagogical approach is and will be eminently practical and collaborative. We want students to not only understand theoretical concepts but also apply them in a real environment. Collaboration among students is essential, fostering teamwork and problem-solving in a practical context.
###### How did you learn about Duckietown?
I learned about Duckietown during a robotics course at the University of Chile. The platform was presented as an innovative educational tool, and I was intrigued by its potential to teach complex concepts in an accessible and attractive way.
###### What is the thing you liked most about using Duckietown in your class?
What I liked most about using Duckietown was the ability to bring abstract concepts to real life. Students were able to see how their algorithms and codes work in a tangible and entertaining environment, making the concepts easier to understand and more appealing.
###### How did your students react to the course?
The students responded very positively.
The practical and playful nature of the course allowed them to actively engage
in learning, and many expressed that the experience with Duckietown has been
one of the most memorable in their education.
Pablo Zapata, a fourth-year student of Computer Engineering and one of the project members, highlights that “my experience has been incredible, as I entered an area that is not part of our curriculum, and for this reason, we have learned to build robots and work with different components. This project has surprised me a lot because of the
technology we are using, and at the same time, it has greatly enriched us over
time.”
On the other hand, Néstor Carvacho, a second-year student in the Computer Programmer Analyst career, points out that “I have been able to work with cutting-edge technology and, with it, learn new work tools that are not part of my career; like Linux, which will help me
in my future work.”
> Duckietown has allowed students from different campuses of the school of computer science and telecommunications at Duoc UC to immerse themselves in the world of robotics and autonomous vehicles, Linux, ROS, and Python, in an accessible and exciting way.
>
> Félix Donoso H.
**How would you recommend we improve the platform?**
Although the platform is excellent, it could benefit from more resources and tutorials aimed at different levels of experience. It would also be helpful to have more examples of projects and applications in different fields to inspire students and showcase the versatility of the platform.
**Would you like to add anything else?**
I would just like to emphasize how valuable the Duckietown platform has been to our institution. It has allowed students from different campuses of the school of computer science and telecommunications at Duoc UC to immerse themselves in the world of robotics and autonomous vehicles, Linux, ROS, and Python, in an accessible and exciting way. I am eager to see how the platform evolves and how we can continue to use it in the future with the practical challenges we are preparing.
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences. It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read other Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** Blog, News, People
**Tags:** autonomous technology, chile, duckiebot, duckietown, DUOC, education, engineering, learn robotics, robotics, robotics education
---
### [How Duckietown set "Frank" Chude Qian on the path to autonomous vehicles](https://duckietown.com/how-discovering-duckietown-set-frank-chude-qian-on-the-path-to-autonomous-vehicles/)
**Published:** February 3, 2023
**Author:** Ivano Marocchi
**Content:**
**University of Toronto, February 3, 2022**: “Frank” Chude Qian, A Master Student at the University of Toronto, shares with us his experience with Duckietown.
##### Quick links
- [ University of Toronto ](https://www.utoronto.ca/)
- [ The University Of Toronto Self-Driving Car Team ](https://www.autodrive.utoronto.ca/)
- [ "Frank" Chude Qian ](https://www.linkedin.com/in/chudeqian/?originalSubdomain=ca)
## How Duckietown set Chude (Frank) Qian on the road to autonomous vehicles
“Frank” Chude Qian is a Master’s student at the University of Toronto. He first encountered Duckietown at the International Conference on Robotics and Automation (ICRA) in 2019, and from that moment he decided that autonomous vehicles would have been his path forward.
**Hello and thank you so much for having accepted to have this brief chat with us! Tell us about yourself and what you do.**
Definitely! My name is Frank, and I’m currently a student at the University of Toronto. I’m in the second year of my program. I started as an Engineering Master’s student, and then I switched over to a Master of Science. My main focus is on developing the second generation of the [University of Toronto’s autonomous vehicles](https://www.autodrive.utoronto.ca/) that participate in the [SAE Auto Drive Challenge](https://autodrivechallenge.com/). My work will end with the transition to the new vehicle, which will happen next year.
**Thank you. Could you describe to us your first approach with Duckietown?**
Yeah, definitely. So I was actually at ICRA 2019 wandering around and figuring out what to do with my life. I was taking part in another competition where I saw the Duckietown setup at the ICRA challenge \[AI-DO 2\]. It looked like great fun, I really loved the idea of how the project is and how it’s designed as a global initiative. You have people from different parts of the world trying to do the same thing, which I found inspiring.
Compared to the actual large-scale autonomous vehicles, Duckietown is an affordable option to learn mobility, and I really liked it. After I came back from ICRA, I just started looking at Duckietown and the AI Driving Olympics competition in more detail.
**Nice! And how did that go?**
In 2019, I started looking into what we could do with Duckiebots at my at [Case Western Reserve University](https://case.edu/), where I was doing my undergrad studies. After learning about the AI-DO challenges, I was like, well, I’ll give it a try! It’s a challenge. I’m pretty competitive. So it took some trying and it ended up, I would say, good enough for where I was back then.
The other thing I looked into was Duckietown’s large code base for demonstrations because I was mainly working on answering the question: “what can you do with limited computing power for a system?”. I tested out the demos, and the Autolab idea, and tried to work on some improvements.
Back in 2019, there wasn’t a lot of work being done on that, unfortunately, but a good part of what was available had been tested, and the documentation was well-proofread. I then took over as a team lead for University of Toronto’s Autonomous Vehicle team, a role less involved with the project, but I still used Duckietown as a great introduction idea, of course.
We have a lot of students who are joining our team with almost no background in autonomous driving, and the Duckietown materials serve as a very good introduction idea to basically educate the younger students on the concept of autonomous vehicles.
Another thing I must say I learned a lot from is the Duckietown challenge and the evaluation side of the AI Driving Olympics, the evaluation server, and the idea of automated evaluation. I think I really gained a lot of experience and knowledge in testing and evaluating thanks to the Duckietown project.
Also along the way, I did another course project on a new baseline for the AI Driving Olympics or for what we call the conditional behavior cloning baseline. So that became another cross-project.
> "It’s not only the cost-effectiveness but also the scalability of Duckietown and the potential it has to make a difference in key industrial sectors of the future."
>
> "Frank" Chude Qian
**It is great to hear that Duckietown helped you get comfortable with real self-driving cars. Is there anything else you would like to add?**
Actually, yes. I know Duckietown is also planning on expanding its materials to the K-12 education side of things. I think that’s a great idea to get more students and younger folks excited about autonomous vehicles. And I think that one day autonomous vehicles will be more and more popular on the road, and the job market for developing and maintaining autonomous vehicles is going to be huge.
I really like the effort, and in fact, that’s probably something I’ll try to do: once I graduate from my current program, I’ll try to hop back on and further the development effort of expanding it to K-12 education.
**Would you recommend Duckietown to students or colleagues?**
Yeah, definitely! And I want to even sort of move it a step further.: for those students who want to get into autonomous vehicle research or development, but maybe their university doesn’t have much funding support for these kinds of programs, Duckietown is such a great project to just adopt.
You just start with the initial concept, and I’ve seen amazing research done thanks to Duckietown. I personally tried a couple of ideas, too. The one thing Duckietown can provide that nothing else can, as far as I can tell, at the same cost range is the development of multi-robot collaboration and the swarm robotics idea.
I think both of these features just provide great advantages for researchers and for students. You know, it’s a Jetson Nano plus some hardware. Or you can use the Raspberry Pi version. I think for students in the universities which don’t provide as much funding this could be a great starting point. And I personally learned so much throughout those projects, and ultimately that lead me to where I am today. So, yeah, definitely I would recommend it.
**What would you say is the biggest quality of Duckietown?**
I think from my experience with the AI-DO, it’s not only the cost effectiveness, but also the scalability of Duckietown and the potential it has to make a difference in key industrial sectors of the future.
**Thank you very much!**
*Note from the editors: a few months after this interview, Frank started working as Software Simulation Developer at General Motors. Congratulations, Frank!*
### Learn more about Duckietown
The Duckietown platform enables state-of-the-art robotics and AI learning experiences.
It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.
[ Find out more use cases ](https://www.duckietown.com/news/people-of-duckietown)
[ Get Started with Duckietown! ](https://www.duckietown.com/guides)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
First Name
Last Name
Email
Type
Story
Subscribe
**Categories:** Blog, News, People
---
### [Announcing the AI Driving Olympics (AI-DO)](https://duckietown.com/announcing-the-ai-driving-olympics/)
**Published:** July 13, 2018
**Author:** Duckietown Admin
**Content:**
[Press release](http://bit.ly/AI-DO-press-release)The Duckietown Foundation is excited to announce the official opening of the **The AI Driving Olympics**, a new competition focused around AI for self-driving cars. The first edition of the AI Driving Olympics 2018 will take place in December 2018, at NIPS, the premiere machine learning conference, in Montréal. This is the first competition that will take place at a machine learning conference with real robots. The second edition of AI-DO is already scheduled to take place in May 2019 in conjunction with the International Conference on Robotics and Automation (ICRA) 2019.
The competition will use the [Duckietown platform](http://www.duckietown.com), a scaled-down affordable and accessible vision-based self-driving car platform used for autonomy education and research. This open-source project originated at MIT in 2016 and is now used by many institutions worldwide.
The AI Driving Olympics is presented in collaboration with 6 academic institutions: ETH Zurich (Switzerland), Université de Montréal (Canada), NCTU (Taiwan), TTIC (USA), Tsinghua (China) and Georgia Tech (USA), as well as two industry co-organizers: nuTonomy and Amazon Web Services (AWS).
The competition will comprise 5 challenges of increasing complexity: 1) Road following on an empty road; 2) Road following with obstacles; 3) Point-to-point navigation in a city network; 4) Point to point navigation in a city network with other vehicles; and 5) Fleet planning for a full autonomous mobility on demand system.
Competitors will have access to simulators, logs, reference implementations, and finally real environments (“Robotariums”) that will be remotely accessible for evaluation. The entries that score best in the robotariums will be run during the live event at NIPS 2018 to determine the winners.
The competition aims at directing academic research towards the hard problems of embodied AI, such as modularity of learning processes, and learning in simulation while deploying in reality. The competition also promotes the democratization of AI/robotics research by offering a common infrastructure available to everybody through the use of remote testing facilities.
Competitors can also build their own Duckiebots using provided DIY instructions, or buy Duckiebots and Duckietown hardware through a [kickstarter campaign](http://bitly.com/DuckietownKickstarter).
For rules and timeline, please see the site https://driving-olympics.ai/
**Categories:** Blog, Events, News
**Tags:** AI Driving Olympics, world
---
### [Duckietown in Ghana – Teaching robotics to brilliant students](https://duckietown.com/duckietown-in-ghana/)
**Published:** August 13, 2018
**Author:** VincentMai
**Content:**
**July 2018**: Vincent Mai travels from Canada to teach a 2-week Duckietown class to some of the brightest high school students in Ghana.
## The email – Montreal, January 2018
On the morning of January 29th, 2018, I received an email. It was a call for international researchers to mentor for two weeks a small group of teenagers that will have been selected among the brightest of Ghana. Robotics was one of the possible topics.
At 4 pm, I had applied.
I was lucky enough to grow up in a part of the world where sciences are available to children. I spent summers in Polytechnique Montreal, playing with electro-magnets and making rockets fly with vinegar and baking soda. I also remember visiting the MIT Museum in Boston, where I was impressed by the bio-inspired swimming robots. There is no doubt that these activities encouraged 17-years-old me to choose physics engineering as my bachelor studies, which then turned into robotics at the graduate level.
### The MISE Foundation
The call from the [MISE Foundation](https://misemaths.org/) was a triple opportunity.
First, I could transmit the passion I was given when I was their age. Second, I would participate, in my small, modest way, in the reduction of education inequalities between developing an developed countries. Countries like Ghana can only benefit from brilliant Ghanaians considering maths, computer science or robotics as a career.
Finally, it was an unique opportunity for me to discover and learn, from people living in an environment that is totally different from mine, with other values, objectives and challenges. It is not everyday you can spend two weeks in Ghana.
After some exchanges with Joel, the organizer, with motivation letters, project plan and visa paperwork, it was decided: I was going to Accra from July 20th to August 6th.
### The preparation – Montreal, June 2018
My specialty is working with autonomous mobile robots: this is what I wanted to teach. I was going to see the brightest young minds of a whole country. I needed to challenge them: I could not go there with a drag-and-drop programmed Lego.
I chose an option that was close to me. Duckietown is a project-based graduate course given at Université de Montréal by my PhD supervisor, Prof. Liam Paull. It allows students to learn the challenges of autonomous vehicles by having miniature cars run in a controlled environment. A Duckiebot is a simple 2-wheel car commanded by a Raspberry Pi. Its only sensor is a camera.
Along with my proximity with Duckietown, I chose it because making a Duckiebot drive autonomously is a very concrete problem, which involves a lot of interesting concepts: computer vision, localization, control, and integration of all these on a controller. Also, for teenagers, the Duckie is a great mascot.
I had not yet taken the Duckietown course. Preparing took me one month and a half of installing, reverse engineering, and documenting. The objective I designed for the kids? Having a Duckiebot named Moose follow the lanes with a constant speed, without getting out of the road or crossing the middle line.
It was inspired from a demo that was already implemented in the Duckiebot. I could not ask the kids to implement the whole code, so I cut out only the most critical parts of it. I also wrote presentations, exercises, planning each of the 10 days we would spend together, 6 hours a day. I packed the sport mats to do the road, a couple of extra pieces in case something broke, and the print-outs of the presentations. I was ready.
Packed Duckietown
Or, I hoped I was. It was not simple to adapt the contents of a graduate course for kids of whom I had no idea of the math and programming level. Did they know how to multiply matrices? What about Bayes law? Can I ask them to use Numpy? When I asked advice to Liam, he told me with a smile: “I guess you’ll have to take the go with the flow…”
### The building – Accra, August 2018
Accra is a large city, spread along the shore of the Atlantic Ocean. Its people are particularly smiling and welcoming. The Lincoln Community School, a private institution hosting the MISE Foundation summer school, has beautiful and calm facilities which allowed us to give the classes in a proper environment. There were 24 children in total: 12 were training for the [International Maths Olympics](https://www.imo-official.org/) with two mentors, while three teams of 4 students would work with a mentor on projects like mine. The two other projects were adversarial attacks on image classifiers and stereo vision.
The first two days, we did maths. I tested their level: they did not know most of what was necessary to go on. Vector operations, integrals, probabilities… We went through these in a very short time: they amazed me by the speed at which they understood.
For the next five days, we went through the project setup. We started simple, understanding how we can drive the Duckiebot with a joystick. We had to setup Moose, discover ROS, and use it to send commands to the motors. We followed with the real project: autonomous mobile robotics.
- See-Think-Act cycle;
- computer vision for line extraction, from RGB images to Canny edge detection and Hough transform;
- camera calibration for ground projection, from image sensors to homography matrix;
- Bayesian estimator for localization, with dynamic prediction and measurement update;
- and finally, proportional control for outputting the right commands to the wheels.
Building Moose
Moose the Duckiebot, up and running!
For each of these steps, the students wrote their version of the code. Then, we made a final version together that we implemented in Moose.
### The experiments – Accra, August 2018
In the two next days, the students had to think what they would do for their research projects. The experiments would be done together but the projects should be individual. Each of them decided to focus on one aspect of autonomous cars. Kwadwo decided to go for speed: he tested the limits of the car as if it was an autonomous ambulance. Abrahim was more concerned about safety: was Moose better than humans at driving? Oheneba thought about the reduction glasshouse gas emissions and William about lowering the traffic. In both cases, they argued that if autonomous cars could improve the situation, they first had to be accepted by humans and therefore be safe and reliable. They tested Moose in differently lit scenes, with white sheets on the road (snow) or with a slightly wrong wheel calibration, to see how it would cope with these conditions.
On the last day, they individually presented their research to a committee formed by the three project mentors. We asked them difficult questions for 15 minutes, testing them and pushing them to think above what they had learned in these 2 weeks. We judged them based on the Intel ISEF criteria (Research project, Methodology, Execution, Creativity and Presentation).
Presenting in front of the judging committee
### The closing ceremony – Accra, August 2018
Saturday was parents day. The students made a general presentation of their projects, making the parents laugh uneasily every time they asked “Is everything clear?” At least, I think most of the parents enjoyed the demonstration: it is always nice to see a Duckiebot run!
Finally, at the closing ceremony, the students who had the best presentation grades were rewarded. I was proud that Kwadwo was named Scholar of the Year, winning a Mobile Robotics book and the right to represent Ghana at the Intel ISEF conference in Phoenix, Arizona, in May 2019. He will present his project with the Duckiebot!
The students and organizers also gave each of us a beautiful gift: a honorary scarf on which it is written “Ayeekoo”. In the local languages, it means: “Job well done.”
I hope I did my job well, and that William, Oheneba, Kwadwo and Abrahim will remember Moose the Duckiebot when they choose their careers. I know that, in any case, these four brilliant young men will continue to shine. On my side, I really enjoyed the experience. I will make sure I don’t miss an opportunity to teach again to teenagers using Duckietown, whether it is in another country or here, in Montreal.
The best team!
### Important note
I had four boys in my group. You can notice on the picture below that, out of the 24 students, only 3 girls participated in the MISE Foundation program. When I asked Joel about it, he told me he has a very difficult time getting women to participate. At least 6 more girls were invited, but their parents would pressure them not to do maths and science, and discourage them from going to the Summer School. They feel this is not what a woman should be doing. I find this situation very frustrating. Ghana is a country with strong family values that are different from the ones I am used to. It is not our role as international researchers to tell them what is good and what is not. And, to be fair, software engineering presents similar ratios in Canada, even if the reasons are less tangible (maybe?).
On the other hand, engineers and scientists build the world around us, and they do so according to the needs they feel. Men cannot build everything women need. I strongly encourage any girl, in any country, who reads this blog post and who is interested about maths and computer science, to stand for what they want to do. We need you here, to build tomorrow’s world together.
MISE 2018 – Ayeekoo!
*You can help the Duckietown Foundation fund similar experiences in Africa and elsewhere in the world by reaching out and donating.*
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!
First Name
Last Name
Email
I am a
Talk about
Reach out
**Categories:** outreach, People
**Tags:** education, outreach, world
---
## Pages
### [Duckietown: learning robotics and AI like the best professionals](https://duckietown.com/)
**Published:** May 24, 2018
**Author:** Duckietown Admin
**Content:**
Teach and Learn Robotics and AI
Create hands-on vehicle autonomy classes and get ready for the era of AI robotics.
[ Get Started ](/guides)
Teach and Learn Robotics and AI
Create hands-on vehicle autonomy classes and get ready for the era of AI robotics.
[ Get Started ](/guides)
Teach and Learn Robotics and AI
Create hands-on vehicle autonomy classes and get ready for the era of AI robotics.
[ Get Started ](/guides)
# [Teach and Learn Robotics and AI](https://duckietown.com/guides)
Create hands-on vehicle autonomy classes and get ready for the era of AI robotics.
[ Get Started ](/guides)
[  ](/guides)
## Teaching robot autonomy
Mastering robotics and AI should not be a struggle.
Born from an MIT first-class experience, Duckietown makes robotics and AI education accessible, effective, and engaging.
We provide streamlined **teaching solutions** for instructors, **hands-on learning experiences** for students, and an **open research platform** for developers.
More than a course, Duckietown is a **comprehensive learning ecosystem** with interactive lectures, real-world simulations, physical robots, rich documentation, and dedicated technical support, bridging theory with practical application.
[](/guides/start-teaching)
[](/guides/start-teaching)
[](/guides/start-teaching)
[](/guides/start-teaching)
[](/guides/start-teaching)
[](/guides/start-teaching)
[](/guides/start-teaching)
[ Start Teaching ](/guides/start-teaching)
## Learning robot autonomy
We want to allow everybody to learn AI and robotics even if they are not at elite institutions like MIT and ETH Zürich.
Duckietown is for learners who want to learn robotics and AI independently, have a deep understanding of how things work, and gain professional skills to boost their careers.
With Duckietown you can build your own robot, follow along with our lectures, and interact with a global community of learners.
[  ](/guides/start-learning)
[ Start Learning ](/guides/start-learning)
## Duckietown for research
Learning robotics and AI is not only for students.
Duckietown has been used extensively for research on mobile robotics and physically embodied AI systems, with Autolabs providing accessible means for reproducible research.
You might be interested in the papers about Duckietown, and learning about how to build your fully customizable smart-city laboratory.
[ Researcher guide ](/guides/start-researching)
## Duckietown around the world
Universities
0 +
Companies
0 +
Countries
0
Teaching the Duckietown class was a wonderful experience for me and my students. The materials are great and the hands-on experience with the robot really helps reinforce the curriculum.
![Matthew R. Walter, Prof.]()
Matthew R. Walter, Prof.Robot Intelligence through Perception Laboratory (RIPL)
Toyota Technological Institute at Chicago
Starting from MIT CSAIL, Duckietown has grown into a global initiative that is inspiring students around the world to learn about self-driving cars, as well as the science and engineering of autonomy.
![Daniela Rus, Prof.]()
Daniela Rus, Prof.Director, Computer Science and AI Lab (CSAIL)
Massachusetts Institute of Technology (MIT)
The Duckietown class is the autonomous driving pie: the filling is hardcore robotics, the casing is artificial intelligence, and as a plus, you get some funny ducks on top!
![Manfred Diaz]()
Manfred DiazPh. D. Candidate
University of Montreal
If University were like learning how to play a new instrument, where lessons are the exercises and exams the final auditions, Duckietown would be the full-blown rock concert, where you play for your fans and look to your heroes with admiration.
![Gioele Zardini]()
Gioele ZardiniPh. D. Candidate
ETH Zurich
Duckietown was much more than just a class, it was a hands-on deep dive into hardware, software, and systems integration, and, most of all, it was a blast!
![Teddy Ort, Ph. D.]()
Teddy Ort, Ph. D.Senior Director
Robot Perception and AI at Symbotic
Spending the summer in Duckietown at MIT made me discover a completely new world: I understood that education can be a game and learning can be fun!
![Valeria Cagnina]()
Valeria CagninaYoung Enterpreneur
Great platform and solid e-learning experience!
![Benjamin Sawicki]()
Benjamin SawickiCoordinator Knowledge & Technology Transfer
NCCR Automation
Highly recommended. I joined the first cohort of "Self-Driving Cars with Duckietown" and loved it.
![Manuel Heredia Ortiz]()
Manuel Heredia OrtizVice President, PhD, Executive MBA
Airbus
Very excited to be a part of the upcoming edition of "Self-Driving Cars with Duckietown". In the last edition, my students and I enjoyed learning of many new topics with hands-on experience. All the student team had a very good experience. Amazing support by Duckietown!
![Kishanprasad Gunale]()
Kishanprasad GunaleAssitant Professor
MIT College of Engineering, Pune
I believe that there’s a lot of interesting research directions that come from a standardized, small scale, accessible autonomous driving platform like Duckietown.
![Liam Paull, Prof.]()
Liam Paull, Prof.Department of Computer Science and Operations Research
Université de Montréal
All the different concepts ranging from control to localization to computer vision can be applied in Duckietown. My students like it and they learn a lot about robotics.
![Francesco Maurelli, Prof.]()
Francesco Maurelli, Prof.Jacobs University of Bremen
In my class we go over the basics of robotics, starting with multi-agent processing or multi-process systems, like most robots are these days, and the Duckiebots are perfect for that
![Paul Robinette, Prof.]()
Paul Robinette, Prof.Department of Electrical and Computer Engineering
University of Massachusetts Lowell
I think Duckietown is a very good education platform for teachers. We make use of the very good materials provided by Duckietown and I’m very satisfied with its implementation.
![Lei Yang, Prof.]()
Lei Yang, Prof.Department of Computer Science and Engineering
University of Nevada, Reno
In engineering, true learning comes from practical implementation, and Duckietown offers that opportunity effectively.
![Shima Akbari]()
Shima AkbariPh. D. Candidate
University of Rome "La Sapienza"
We have received large positive feedback for Duckietown from students and researchers who have used it at IIT Jodhpur, with many appreciating its entertaining and challenging nature.
![Debasis Das, Prof.]()
Debasis Das, Prof.IIT Jodhpur
Duckietown has allowed students from different campuses of the school of computer science and telecommunications at Duoc UC to immerse themselves in the world of robotics and autonomous vehicles, Linux, ROS, and Python, in an accessible and exciting way.
![Félix Donoso H., Prof.]()
Félix Donoso H., Prof.School of Computer Science and Telecommunications at the Duoc UC Professional Institute
I haven't found another hardware platform as good as Duckietown for my needs. It's a simple platform with a fun approach, well-documented, and with a really reactive community. Even compared to other commercial products, Duckieown stands out. It fulfills all the needs from beginners to experts.
![Peter Affolter, Prof.]()
Peter Affolter, Prof.Berner Fachhochschule (BFH)
[  ](https://www.uniroma1.it/it/pagina-strutturale/home)
[  ](https://www.umontreal.ca/en/)
[  ](https://constructor.university/)
[  ](https://ethz.ch/en.html)
[  ](https://www.uml.edu/)
[  ](https://etu.ru/en/university/)
[  ](https://www.duoc.cl/)
[  ](https://www.bfh.ch/de/)
[  ](https://www.umontpellier.fr/)
[  ](https://www.ualberta.ca/en/index.html)
[  ](https://constructor.university/)
[  ](https://www.grenoble-inp.fr/)
[  ](https://www.uml.edu/)
[  ](https://www.ox.ac.uk/)
[  ](https://web.uniroma2.it/en)
[  ](https://www.bme.hu/en)
[  ](https://www.uao.edu.co/)
[  ](https://www.utoronto.ca/)
[  ](https://www.bme.hu/en)
[  ](https://www.bu.edu/)
[  ](https://www.unr.edu/)
[  ](https://iitj.ac.in/)
[  ](https://www.startpage.com/sp/search)
[  ](https://www.tum.de/)
[  ](https://charleston.edu/)
[  ](https://www.agh.edu.pl/en/)
[  ](https://tu-dresden.de/?set_language=en)
[  ](https://www.massrobotics.org/)
[  ](https://www.uct.ac.za/)
[  ](https://www.hacettepe.edu.tr/english)
## People, papers and projects in Duckietown
[](https://duckietown.com/duckie-day-2026-salmon-farming/)## [ Duckie Day 2026 brings robotics to salmon farming ](https://duckietown.com/duckie-day-2026-salmon-farming/)
[](https://duckietown.com/teaching-robot-autonomy-the-hague-university/)## [ Teaching robot autonomy at The Hague University ](https://duckietown.com/teaching-robot-autonomy-the-hague-university/)
[](https://duckietown.com/robotics-rome-cup-2026/)## [ Rome Cup 2026 ](https://duckietown.com/robotics-rome-cup-2026/)
[](https://duckietown.com/real-time-reinforcement-learning-in-duckiematrix/)## [ Real-Time Reinforcement Learning in Duckiematrix ](https://duckietown.com/real-time-reinforcement-learning-in-duckiematrix/)
[](https://duckietown.com/db21v3-j-duckiebot-upgrade-kit-is-out/)## [ DB21v3-J Duckiebot upgrade kit now available ](https://duckietown.com/db21v3-j-duckiebot-upgrade-kit-is-out/)
[](https://duckietown.com/duckiebot-db21j-now-available-pre-assembled-and-initialized/)## [ Duckiebots now available pre-assembled and initialized ](https://duckietown.com/duckiebot-db21j-now-available-pre-assembled-and-initialized/)
« PreviousPage1[Page2](https://duckietown.com/wp-cron.php/page/2/?doing_wp_cron=1786646276.4517979621887207031250)[Page3](https://duckietown.com/wp-cron.php/page/3/?doing_wp_cron=1786646276.4517979621887207031250)…[Page26](https://duckietown.com/wp-cron.php/page/26/?doing_wp_cron=1786646276.4517979621887207031250)[Next »](https://duckietown.com/wp-cron.php/page/2/?doing_wp_cron=1786646276.4517979621887207031250)
[ People ](/news/people-of-duckietown/)
[ Papers ](/research-papers/)
[ Projects ](/projects-for-learning-robotics-and-ai/)
## Stay in touch
Sign up to our newsletter if you want to receive occasional updates.
---
### [The Impact of Autonomous Drone Technology](https://duckietown.com/the-impact-of-autonomous-drone-technology/)
**Published:** July 3, 2025
**Author:** Duckietown Admin
**Content:**
# The Impact of Autonomous Drone Technology
They are relatively inexpensive, can be built from commonly available components, can be very small and light or large and heavy, can pack customizable payloads from fancy sensors to releasable matter, they are programmable, they *fly*, and now are even becoming autonomous. How could autonomous drone technology not make a difference?
With autonomous drone technology, we refer to a set of hardware and software systems designed and deployed to enable unmanned operations for aerial vehicles (UAVs), i.e., to perform tasks without, or significantly reduced, depending on the applications, direct human input.
These systems integrate onboard sensors, actuators, compute (processors), and perception, planning, and control algorithms to operate with agency in structured and unstructured environments. While the technology in itself is not necessarily novel, the versatility and speed of small flying platforms, now particularly accessible on the market, expose diverse applications: from surveillance, inspection, entertainment, and defense.
##### Table of Contents
- [ What is an autonomous drone ](#chap2)
- [ How do they work? ](#chap3)
- [ Applications of autonomous drones ](#chap4)
- [ The future of autonomous drones ](#chap5)
- [ The role of Duckietown in drone education ](#chap6)
- [ Start building your own autonomous quadcopter ](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24)
## Introduction to autonomous drone technology
The first autonomous systems were developed for “dull, dirty, or dangerous” tasks that no human would want to be involved with.
As with many technological advancements throughout history, the application field for the first aerial autonomous systems was the military. Surely, deploying technology to substitute humans and enable, at least, remote operation was more cost-effective and safer (for the pilot) to train a top gun.
The evolution of autonomous drone technology progressed from unmanned balloons driven by wind currents, to satellites opening the doors to the space race, to manually piloted hobbyist quadcopter models (FPV drones), to fully autonomous systems used in defense, research, and industry.
Early UAVs relied on remote control and line-of-sight operation. These gave way to DIY (Do-It-Yourself) platforms with limited autonomy, followed by commercial drones with partial autonomy features. High-cost military-grade UAVs introduced onboard navigation and mission logic, setting the foundation for scalable autonomous systems.
Today, autonomous drones are commercially available at a lower cost and with greater functionality.
### What is an autonomous drone?
**An autonomous drone is a robot that flies.**
Also known as an unmanned aerial vehicle (UAV), it integrates:
- perception (sensors and cameras)
- planning (decision-making algorithms)
- control (flight stabilization and navigation systems)
After mission initialization, it uses real-time data to adapt, navigate, and complete tasks such as mapping, inspection, or delivery with minimal external input.
### How does an autonomous drone work?
A drone becomes autonomous through the integration of sensing, planning, computation, and control systems. This architecture mirrors the general robotic autonomy framework as used in Duckietown’s Duckiebots.
1. **Perception**
Autonomous drones rely on GPS, inertial measurement units (IMUs), cameras, and LiDAR for localization, stability, and environment mapping. Visual SLAM algorithms process visual input to build real-time maps, while sensor fusion improves positional accuracy. Obstacle detection systems identify and respond to hazards dynamically.
2. **Planning**
Onboard processors execute autonomy logic, enabling task-level decision-making such as waypoint navigation, path planning, and contingency handling. AI algorithms, including machine learning models, support behavior adaptation in uncertain or changing environments.
3. **Control**
Autonomous flight is maintained through closed-loop controllers (PID or MPC) that stabilize attitude and velocity. These controllers respond to sensor feedback with low latency, ensuring precise tracking of planned trajectories.
All components are managed on-device or via distributed systems, allowing near real-time, closed-loop autonomy in both simulation and physical environments.
### Applications of autonomous drone technology
Autonomous drone technology is now embedded across multiple sectors due to its ability to operate efficiently in complex environments with minimal human input. In disaster response, autonomous drones are deployed for search-and-rescue, thermal imaging, and terrain mapping, such as in the aftermath of the 2023 Turkey earthquake. In agriculture, drones are used for crop health monitoring, variable-rate spraying, and field analytics, improving yield while minimizing resource usage.
High-resolution aerial mapping enables access to remote or previously unreachable areas, supporting urban planning, mining, and environmental monitoring. Surveillance applications span public safety, wildlife conservation, and infrastructure security. Law enforcement and military units use autonomous drones for reconnaissance, perimeter control, and battlefield situational awareness widely observed during the COVID-19 pandemic[.](https://pmc.ncbi.nlm.nih.gov/articles/PMC9612140/ "COVID-19 pandemic")
Autonomous drones also support industrial inspection tasks such as monitoring bridges, pipelines, and transmission lines. Outside functional domains, drones are being integrated into entertainment, with autonomous swarms enabling large-scale aerial light shows.
#### Surveillance and security
Autonomous drones are adopted in surveillance and security due to their ability to operate with minimal human oversight and efficiently cover large or inaccessible areas. Equipped with high-resolution cameras, thermal sensors, and onboard AI, these systems provide continuous monitoring of borders, critical infrastructure, and urban zones.
They enable rapid detection of anomalies, unauthorized access, and security breaches, supporting law enforcement and emergency response with actionable intelligence. Autonomous flight paths, geofencing, and automated alerting enhance situational awareness and reduce response latency.
#### Mapping and surveying
Autonomous drones are redefining mapping and surveying by enabling fast, high-resolution, and repeatable data collection across varied and complex terrains. Equipped with GPS, LiDAR, and optical or multispectral imaging systems, these drones generate accurate 2D orthomosaics and 3D models for use in construction, agriculture, environmental analysis, and urban planning.
Autonomous flight along predefined routes ensures consistent data capture over time, enabling longitudinal studies and frequent map updates. This approach reduces manual labor, improves safety in inaccessible or hazardous zones, and lowers operational costs while increasing spatial and temporal data resolution supporting more informed, real-time decision-making across industries.
#### Equipment inspection and maintenance
Autonomous drones are transforming infrastructure and equipment inspection by enabling safe, efficient, and high-precision assessments of assets such as power lines, pipelines, wind turbines, and bridges. Using high-resolution cameras, thermal sensors, and LiDAR systems, these drones identify structural anomalies including cracks, corrosion, thermal hotspots, and mechanical degradation.
Autonomous flight paths allow for consistent, repeatable inspections without the need for scaffolding, rope access, or shutdowns. This reduces operational risk, lowers inspection costs, and minimizes downtime while improving data accuracy and maintenance scheduling. Autonomous drones play a vital role in predictive maintenance and asset lifecycle management by safely reaching confined or hazardous areas, allowing for detailed inspections and data collection without interrupting regular operations.
#### Disaster response
Autonomous drones are critical tools in disaster response operations, offering rapid situational awareness, real-time data collection, and logistical support in environments that are often inaccessible or unsafe for human responders. Outfitted with thermal imaging, environmental sensors, and live communication links, these systems assist in locating survivors, assessing structural damage, mapping affected zones, and delivering essential supplies.
Pre-programmed flight plans and autonomous navigation enable consistent coverage of target areas under time-sensitive and hazardous conditions. Their deployment accelerates decision-making, improves coordination among emergency services, and enhances overall response efficiency reducing human risk and supporting faster recovery operations.
#### Agriculture
Autonomous drones are advancing agricultural operations through enhanced crop monitoring, precision farming, and data-driven decision-making. Using multispectral and thermal imaging sensors, these drones capture high-resolution imagery to assess plant health, soil moisture, nutrient levels, and early signs of pest infestation or disease.
Autonomous flight paths enable rapid, repeatable coverage of large areas, facilitating targeted interventions such as variable-rate irrigation, fertilization, and pesticide application. This approach improves yield, optimizes resource utilization, reduces environmental impact, and lowers labor requirements supporting scalable, efficient farm management.
#### Military operations
Autonomous drones are widely integrated into modern military operations, serving roles in surveillance, reconnaissance, target acquisition, and precision strike execution. Their autonomous navigation capabilities enable extended missions with minimal operator input, reducing risk to personnel and maintaining persistent situational awareness over contested or inaccessible areas.
Outfitted with electro-optical sensors, infrared imaging, and optionally, guided munitions, these systems provide real-time intelligence, monitor enemy movement, and support tactical decision-making. Autonomous drones offer a scalable, cost-effective alternative to manned assets, contributing to both defensive posture and offensive operations in multi-domain environments.
#### Delivery
Autonomous drones are reshaping logistics by enabling fast, cost-efficient, and scalable delivery of goods. Using GPS, inertial navigation systems, and real-time obstacle avoidance, these drones execute short- to medium-range missions for transporting packages, medical supplies, and food without human intervention.
Autonomous routing allows for bypassing ground traffic and accessing remote or congested urban zones where conventional delivery is constrained. This reduces delivery time, lowers labor costs, and minimizes carbon emissions compared to traditional vehicles. As regulatory frameworks and technologies mature, autonomous drone delivery is positioned to become a core element of last-mile logistics and e-commerce fulfillment.
#### The future of autonomous drones
The future of autonomous drone technology is expected to significantly impact transportation, infrastructure, logistics, and public services. In smart cities, autonomous drones may support real-time traffic monitoring, environmental data collection, infrastructure inspection, and emergency response coordination. Drone-based delivery systems could become standard in logistics, enabling rapid, contactless transport of goods across urban and remote areas.
However, the proliferation of autonomous drones also introduces challenges related to airspace management, privacy, cybersecurity, and ethical use particularly as surveillance and data collection capabilities expand. Public trust and societal acceptance will depend on transparent governance and responsible deployment.
Regulatory frameworks are currently evolving to address these concerns, aiming to balance innovation with operational safety, accountability, and legal compliance. While continued advancements in AI, autonomy, and battery systems will accelerate development, the exact trajectory of adoption will depend on technological maturity, economic viability, and policy alignment at national and global levels.
### The role of Duckietown in autonomous drone technology
Duckietown, through its Duckiedrone platform, aims to make autonomous drone technology accessible for education and research. Designed for modularity and ease of use, the Duckiedrone serves as a practical entry point for learners to study the core components of autonomy including perception, control, and AI integration using real hardware and open-source tools.
For educators, the platform provides a structured and scalable educational drone platform to teach robotics, computer vision, and autonomous systems in both physical and simulated environments. Its hands-on approach bridges theoretical learning and real-world implementation.
By lowering technical and financial barriers, Duckietown democratizes access to drone autonomy, allowing students, researchers, and hobbyists to engage with the same principles used in industrial applications such as smart infrastructure, logistics, and aerial surveillance. The open design encourages experimentation and extension, supporting both foundational learning and advanced prototyping.
[ Get your Duckiedrone ](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24)
### Learn more about Duckietown
The [Duckietown](https://duckietown.com/ "Duckietown website main page") platform enables state-of-the-art [robotics and AI learning experiences](https://github.com/duckietown/duckietown-lx "Duckietown learning experiences (LX)").
It is designed to help [teach](https://duckietown.com/guides/start-teaching/ "Duckietown starter guide for teachers"), [learn](https://duckietown.com/guides/start-learning/ "Duckietown starter guide for learners"), and [do research](https://duckietown.com/guides/start-researching/ "Duckietown starter guide for researchers"): from exploring the [fundamentals of computer science and automation](https://duckietown.com/educational-resources/ "Duckietown education materials") to [pushing the boundaries of human knowledge](https://duckietown.com/research-papers/ "Duckietown research papers").
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read the Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
---
### [Best Robotics and AI Projects](https://duckietown.com/best-robotics-and-ai-projects-for-learning-autonomy/)
**Published:** April 26, 2024
**Author:** Duckietown Admin
**Content:**
# Robotics and AI Projects
**This is a collection of robotics and AI projects for learning robot autonomy hands-on, mostly picked from Duckietown-related university courses worldwide.**
Creating novel autonomous behaviors for your Duckiebots, or working to improve upon existing ones, is a great way to learn how to work in a team and gain real-world skills in the process. Let us know about your robotics and AI projects!
[ Share your project ](#projects-reach-out)
## Robotics an AI project ideas
[
](https://duckietown.com/real-time-reinforcement-learning-in-duckiematrix/)##### [ Real-Time Reinforcement Learning in Duckiematrix ](https://duckietown.com/real-time-reinforcement-learning-in-duckiematrix/)
This project explores Reinforcement Learning in Duckiematrix within Duckietown, analyzing real-time delays and their impact on autonomous driving performance.
[ Read More » ](https://duckietown.com/real-time-reinforcement-learning-in-duckiematrix/)
[
](https://duckietown.com/sim-to-sim-to-real-transfer-for-small-autonomous-vehicles/)##### [ Sim-to-Sim-to-Real Transfer for Small Autonomous Vehicles ](https://duckietown.com/sim-to-sim-to-real-transfer-for-small-autonomous-vehicles/)
This project studies Sim-to-Real Transfer in Duckietown using high- and low-fidelity simulators to predict autonomous vehicles performance on Duckiebots.
[ Read More » ](https://duckietown.com/sim-to-sim-to-real-transfer-for-small-autonomous-vehicles/)
[
](https://duckietown.com/autoduck-vlm-autonomous-navigation-in-duckietown/)##### [ Autoduck: VLM-based Autonomous Navigation in Duckietown ](https://duckietown.com/autoduck-vlm-autonomous-navigation-in-duckietown/)
This project implements an autonomous navigation system on the Duckiebot DB21J in the Duckietown environment using vision-based control and VLMs.
[ Read More » ](https://duckietown.com/autoduck-vlm-autonomous-navigation-in-duckietown/)
[
](https://duckietown.com/autonomous-navigation-in-duckietown-with-quackcruiser/)##### [ QuackCruiser: Autonomous Navigation with Dijkstra ](https://duckietown.com/autonomous-navigation-in-duckietown-with-quackcruiser/)
This project develops Duckietown Autonomous Navigation using Dijkstra planning, perception, and control for lane following, turning, and obstacle detection.
[ Read More » ](https://duckietown.com/autonomous-navigation-in-duckietown-with-quackcruiser/)
[
](https://duckietown.com/localization-with-sensor-fusion-in-duckietown/)##### [ Duckiebot Localization with Sensor Fusion in Duckietown ](https://duckietown.com/localization-with-sensor-fusion-in-duckietown/)
This project explores localization in Duckietown using sensor fusion to estimate Duckiebot poses through visual tags and stitched camera inputs.
[ Read More » ](https://duckietown.com/localization-with-sensor-fusion-in-duckietown/)
[
](https://duckietown.com/visual-feedback-for-autonomous-navigation-in-duckietown/)##### [ Features for Efficient Autonomous Navigation in Duckietown ](https://duckietown.com/visual-feedback-for-autonomous-navigation-in-duckietown/)
Students at TUM build on the Duckietown out-of-the-box autonomous navigation pipeline introducing features for a more complete driving experience.
[ Read More » ](https://duckietown.com/visual-feedback-for-autonomous-navigation-in-duckietown/)
[
](https://duckietown.com/pure-pursuit-lane-following-with-obstacle-avoidance/)##### [ Pure Pursuit Lane Following with Obstacle Avoidance ](https://duckietown.com/pure-pursuit-lane-following-with-obstacle-avoidance/)
This project implements adaptive pure pursuit control and deep learning-based obstacle detection for lane following and obstacle avoidance for Duckiebots.
[ Read More » ](https://duckietown.com/pure-pursuit-lane-following-with-obstacle-avoidance/)
[
](https://duckietown.com/autonomous-navigation-system-development-in-duckietown/)##### [ Autonomous Navigation System Development in Duckietown ](https://duckietown.com/autonomous-navigation-system-development-in-duckietown/)
This project implements an Autonomous Navigation System using computer vision, and Dijkstra algorithm for precise lane following and safe intersection handling.
[ Read More » ](https://duckietown.com/autonomous-navigation-system-development-in-duckietown/)
[
](https://duckietown.com/autonomous-navigation-and-parking-in-duckietown/)##### [ Autonomous Navigation and Parking in Duckietown ](https://duckietown.com/autonomous-navigation-and-parking-in-duckietown/)
This project uses PID control, AprilTag-based turns, dead reckoning, and visual servoing to enable autonomous navigation and parking in Duckietown.
[ Read More » ](https://duckietown.com/autonomous-navigation-and-parking-in-duckietown/)
[
](https://duckietown.com/extended-kalman-filter-slam-for-duckiebots/)##### [ Extended Kalman Filter (EKF) SLAM for Duckiebots ](https://duckietown.com/extended-kalman-filter-slam-for-duckiebots/)
Implementing Extended Kalman Filter (EKF) SLAM on Duckiebots to enhance localization accuracy and map static landmarks using AprilTags and odometry.
[ Read More » ](https://duckietown.com/extended-kalman-filter-slam-for-duckiebots/)
[
](https://duckietown.com/path-planning-for-multi-robot-navigation-in-duckietown/)##### [ Path Planning for Multi-Robot Navigation in Duckietown ](https://duckietown.com/path-planning-for-multi-robot-navigation-in-duckietown/)
A path planning algorithm that enables optimized multi-robot navigation in Duckietown, using nodegraph mapping and movement minimization techniques.
[ Read More » ](https://duckietown.com/path-planning-for-multi-robot-navigation-in-duckietown/)
[
](https://duckietown.com/city-rescue-autonomous-recovery-system-for-duckiebots/)##### [ City Rescue: Autonomous Recovery System for Duckiebots ](https://duckietown.com/city-rescue-autonomous-recovery-system-for-duckiebots/)
Learn how to transform Duckietown in a smart city performing autonomous recovery of distressed Duckiebots, through vehicle to infrastructure (v2i) interactions.
[ Read More » ](https://duckietown.com/city-rescue-autonomous-recovery-system-for-duckiebots/)
[
](https://duckietown.com/adaptive-lane-following-with-automatic-trim-calibration/)##### [ Adaptive Lane Following with Auto-Trim Tuning ](https://duckietown.com/adaptive-lane-following-with-automatic-trim-calibration/)
This project enhances Adaptive Lane Following by enabling Duckiebots to autonomously calibrate wheel trim, ensuring stable navigation without manual tuning.
[ Read More » ](https://duckietown.com/adaptive-lane-following-with-automatic-trim-calibration/)
[
](https://duckietown.com/flexible-tether-control-in-heterogeneous-marsupial-systems/)##### [ Flexible tether control in marsupial systems ](https://duckietown.com/flexible-tether-control-in-heterogeneous-marsupial-systems/)
This project develops a flexible tether control system for Duckiebot, using ROS and an automated spool to optimize tether length for mobility and efficiency.
[ Read More » ](https://duckietown.com/flexible-tether-control-in-heterogeneous-marsupial-systems/)
[
](https://duckietown.com/deep-reinforcement-learning-for-autonomous-lane-following/)##### [ Deep Reinforcement Learning for Autonomous Lane Following ](https://duckietown.com/deep-reinforcement-learning-for-autonomous-lane-following/)
This project uses deep reinforcement learning for autonomous lane following, tackling sim-to-real challenges with domain adaptation and vision-based control.
[ Read More » ](https://duckietown.com/deep-reinforcement-learning-for-autonomous-lane-following/)
[
](https://duckietown.com/visual-obstacle-detection-using-inverse-perspective-mapping/)##### [ Visual Obstacle Detection using Inverse Perspective Mapping ](https://duckietown.com/visual-obstacle-detection-using-inverse-perspective-mapping/)
This project develops a visual obstacle detection system in Duckietown using inverse perspective mapping to improve autonomous navigation accuracy.
[ Read More » ](https://duckietown.com/visual-obstacle-detection-using-inverse-perspective-mapping/)
[
](https://duckietown.com/intersection-navigation-in-duckietown-using-3d-image-feature/)##### [ Intersection Navigation in Duckietown Using 3D Image Features ](https://duckietown.com/intersection-navigation-in-duckietown-using-3d-image-feature/)
Explore how 3D image features enhance Duckiebot intersection navigation in Duckietown, blending cutting-edge BEV tech with hands-on autonomous driving insights.
[ Read More » ](https://duckietown.com/intersection-navigation-in-duckietown-using-3d-image-feature/)
[
](https://duckietown.com/monocular-navigation-in-duckietown-using-lednet-architecture/)##### [ Monocular Navigation in Duckietown Using LEDNet Architecture ](https://duckietown.com/monocular-navigation-in-duckietown-using-lednet-architecture/)
This project is on monocular navigation in Duckietown using LEDNet, comparing its performance with vision transformer for lane-following and obstacle avoidance.
[ Read More » ](https://duckietown.com/monocular-navigation-in-duckietown-using-lednet-architecture/)
[
](https://duckietown.com/reinforcement-learning-for-the-control-of-autonomous-robots/)##### [ Reinforcement Learning for the Control of Autonomous Robots ](https://duckietown.com/reinforcement-learning-for-the-control-of-autonomous-robots/)
This thesis applies Reinforcement Learning (RL) for autonomous lane-keeping and YOLO v5 obstacle detection in Duckietown, achieving safe navigation.
[ Read More » ](https://duckietown.com/reinforcement-learning-for-the-control-of-autonomous-robots/)
[
](https://duckietown.com/smart-lighting-autonomous-driving-realistic-day-and-night-in-duckietown/)##### [ Smart Lighting: Realistic Day and Night in Duckietown ](https://duckietown.com/smart-lighting-autonomous-driving-realistic-day-and-night-in-duckietown/)
What if Duckietowns had smart lighting, so that car and street light fields would combine dynamically for optimal visual perception?
[ Read More » ](https://duckietown.com/smart-lighting-autonomous-driving-realistic-day-and-night-in-duckietown/)
[
](https://duckietown.com/dbscan-driven-intersection-navigation-for-duckiebots/)##### [ Duckiebot Intersection Navigation with DBSCAN ](https://duckietown.com/dbscan-driven-intersection-navigation-for-duckiebots/)
This project uses DBSCAN (Density-Based Algorithm for Discovering Clusters
in Large Spatial Databases with Noise) to improve Duckiebot intersection navigation.
[ Read More » ](https://duckietown.com/dbscan-driven-intersection-navigation-for-duckiebots/)
[
](https://duckietown.com/obstavoid-dynamic-obstacle-avoidance-in-duckietown/)##### [ Obstacle Avoidance for Dynamic Navigation Using Obstavoid ](https://duckietown.com/obstavoid-dynamic-obstacle-avoidance-in-duckietown/)
The Obstavoid Algorithm enables obstacle avoidance in Duckietown in realtime, calculating optimal paths using a 3D grid for dynamic, collision-free navigation.
[ Read More » ](https://duckietown.com/obstavoid-dynamic-obstacle-avoidance-in-duckietown/)
[
](https://duckietown.com/protip-duckiebot-remote-connection/)##### [ ProTip: Duckiebot Remote Connection ](https://duckietown.com/protip-duckiebot-remote-connection/)
Have you ever wanted to work from home but your Duckiebot is back at the lab? Learn how to access your Duckiebot from anywhere at any time.
[ Read More » ](https://duckietown.com/protip-duckiebot-remote-connection/)
[
](https://duckietown.com/monocular-visual-odometry-for-duckiebot-navigation/)##### [ Monocular Visual Odometry for Duckiebot Navigation ](https://duckietown.com/monocular-visual-odometry-for-duckiebot-navigation/)
This project by Gianmarco Bernasconi, a former Duckietown student, provided an estimate of the Duckiebot’s pose using a monocular visual odometry approach.
[ Read More » ](https://duckietown.com/monocular-visual-odometry-for-duckiebot-navigation/)
[
](https://duckietown.com/goto-1-planning-with-dijkstra/)##### [ Goto-1: Planning with Dijkstra ](https://duckietown.com/goto-1-planning-with-dijkstra/)
This project enhances Duckiebot planning capabilities for autonomous navigation in Duckietowns using the Dijkstra algorithm.
[ Read More » ](https://duckietown.com/goto-1-planning-with-dijkstra/)
[
](https://duckietown.com/yolo-based-robust-object-detection/)##### [ YOLO-based Robust Object Detection in Duckietown ](https://duckietown.com/yolo-based-robust-object-detection/)
This project implements robust object detection in Duckietown for Duckiebots under varying lighting conditions and object clutter using a YOLO-based NN.
[ Read More » ](https://duckietown.com/yolo-based-robust-object-detection/)
[
](https://duckietown.com/dynamic-obstacle-avoidance-in-duckietown/)##### [ Implementing vision based dynamic obstacle avoidance ](https://duckietown.com/dynamic-obstacle-avoidance-in-duckietown/)
This student project implements dynamic obstacle avoidance for Duckiebots with the aim of detecting and navigating around static and moving obstacles.
[ Read More » ](https://duckietown.com/dynamic-obstacle-avoidance-in-duckietown/)
[
](https://duckietown.com/ackermann-steering-duckiebot/)##### [ Development of an Ackermann steering autonomous vehicle ](https://duckietown.com/ackermann-steering-duckiebot/)
This student project implements an Ackermann steering system on a Duckiebot to simulate 4 wheels, and better and more complex real-world car model.
[ Read More » ](https://duckietown.com/ackermann-steering-duckiebot/)
[
](https://duckietown.com/autonomous-parking-in-duckietown/)##### [ Introducing Autonomous Parking in Duckietown Cities ](https://duckietown.com/autonomous-parking-in-duckietown/)
This student project implements an autonomous parking solution, inclusive of parking lot design and autonomous behavior, for Duckiebots in Duckietown.
[ Read More » ](https://duckietown.com/autonomous-parking-in-duckietown/)
[
](https://duckietown.com/safe-reinforcement-learning-rl-duckietown-thesis-project/)##### [ Safe Reinforcement Learning (RL) Thesis Project ](https://duckietown.com/safe-reinforcement-learning-rl-duckietown-thesis-project/)
“Safe Reinforcement Learning (Safe-RL)” explores using Deep Q Learning to train Duckiebots to perform lane following. Reproduce these results with Duckietown.
[ Read More » ](https://duckietown.com/safe-reinforcement-learning-rl-duckietown-thesis-project/)
[
](https://duckietown.com/anatidaephilia-centralized-city-based-slam-cslam/)##### [ Anatidaephilia: centralized city-based SLAM (cSLAM) ](https://duckietown.com/anatidaephilia-centralized-city-based-slam-cslam/)
Anatidaephilia is loving the idea that somewhere, somehow, a duck is watching you. The cSLAM equips Duckietowns with the ability to localize Duckiebots.
[ Read More » ](https://duckietown.com/anatidaephilia-centralized-city-based-slam-cslam/)
### Duckietown projects for learning robotics to share? Let us know!
If you would like your project to be featured on this page, let us know about it!
---
### [Pricing for research and enterprise](https://duckietown.com/pricing/researchers-and-enterprise/)
**Published:** April 21, 2026
**Author:** Duckietown Admin
**Content:**
# Robotics and AI solutions for research and enterprise
###### Duckietown supports robotics research, autonomy development, embodied AI experimentation, and workforce training through modular hardware, open software, simulation environments, and deployment support.
## Who is this for?
Duckietown is a robotics research platform with research and enterprise solutions designed for:
- university research laboratories
- robotics and AI research groups
- industrial R&D teams
- workforce upskilling programs
- autonomy and embodied AI experimentation
- robotics innovation initiatives
who are looking to perform reproducible robotics and embodied AI experimentation, robotics research, autonomy education curricula development, fleet robotics, multi-modal scenarios, sim-to-real R&D, reinforcement learning, or other simulation-based development.
If you are not affiliated with a university or company, check out the [independent personal learning](https://duckietown.com/pricing/plans-for-learners/ "Duckietown independent learner license requirements") page.
For activities mostly focused on academic teaching and learning, see [Duckietown licensing for universities and schools](https://duckietown.com/pricing/universities-and-schools/ "Duckietown plans for universities and schools").
### Open autonomy
platform
Access to Duckietown software, simulation,
and development tools
- Open autonomy stack
- Duckietown Shell (CLI)
- Duckietown desktop applications (GUI)
- Duckiematrix virtual environment
- Virtual Duckiebots and Duckiedrones
- Duckietown library
- Developer and research community access
- Cross-platform support
- Containerized workflows
- ROS/ROS2-compatible autonomy workflows
[ Contact Us ](https://duckietown.com/request-quote/)
### Technical
Support
Advanced onboarding and
deployment support
- Onboarding for laboratory staff
- Staff priority support
- Private support channel
- Lab channel in Duckietown community
- Priority fulfillment
- Early access to new features
[ Contact Us ](https://duckietown.com/request-quote/)
### Robotics hardware
ecosystem
Equipment available
worldwide
$ 399+
- Modular hardware platform
- Open and extensible autonomy software
- Modular robotic agents
- ROS, ROS2 support
- Self-driving cars
- Smart-city robotics environment
- Autonomous quadcopter drones
- Powered by Raspberry Pi and NVIDIA Jetson platforms
- Browser-based diagnostic dashboards
- 6-month hardware warranty
- 30-day return policy
- Worldwide shipping
[ Contact Us ](https://duckietown.com/request-quote/) Laboratory bundles and optional pre-assembly and initialization of Duckiebots available
### Frequently asked questions (FAQ)
##### Q. Can Duckietown be customized for research?
Yes, researchers worldwide have published [hundreds of peer-reviewed papers leveraging Duckietown](https://duckietown.com/research-papers/ "Research with Duckietown").
Duckietown supports customizable robotics and AI research workflows, including autonomous driving, fleet coordination, embodied AI, reinforcement learning, computer vision, and simulation-based experimentation.
Researchers can leverage the mature designs of Duckietown mobile robots to get started quickly, and augment the hardware builds with additional sensors and actuators thanks to the open design and modular, containerized software architecture. Baseline autonomy pipelines are provided out of the box for rapid deployment.
Custom integrations and research-oriented support arrangements may also be available for larger deployments or collaborative projects.
##### Q. Is the platform open source?
Duckietown provides an open and extensible robotics and AI development platform designed for education, research, and experimentation.
The autonomy software stack running on Duckietown robots is openly accessible and designed to support customization, research, and development workflows.
Some infrastructure services, platform components, and deployment-oriented tools within the broader Duckietown ecosystem may remain proprietary.
##### Q. Can Duckietown integrate with ROS (and ROS2)?
Yes.
Duckietown supports robotics development workflows compatible with ROS and related robotics software ecosystems. The out-of-the-box autonomy pipelines for Duckiebots and Duckiedrones are ROS-based. ROS2 agents are supported too.
The platform is designed to support experimentation, integration, and extension within modern robotics and autonomy research environments.
##### Q. Is simulation available?
Yes.
Duckietown includes the Duckiematrix virtual environment and virtual robot workflows for simulation-based robotics development and experimentation. The Duckiematrix supports hybrid “hardware in the loop” experiments, too.
Simulation can be used for autonomy development, testing, coursework, reinforcement learning, and fleet experimentation before deploying to physical robots.
##### Q. Do you support hardware customizations?
In some cases, yes.
Duckietown’s modular robotics platform allows institutions and organizations to adapt hardware configurations depending on research goals, deployment constraints, or educational requirements, thanks to open analog and digital ports on our robots.
Custom hardware arrangements, fleet configurations, and deployment-oriented modifications may be available upon request.
##### Q. Can Duckietown support fleet experiments?
Yes.
Duckietown is designed to support multi-robot and fleet robotics experimentation, including autonomous driving coordination, traffic management, distributed robotics, and smart-city research scenarios.
The platform supports both simulated and physical robotics environments.
##### Q. Is deployment support available?
Yes.
Duckietown can support onboarding, deployment planning, laboratory setup, fleet initialization, and technical guidance depending on the scope of the project and support arrangement.
Priority support options may also be available for research laboratories, institutional deployments, and enterprise collaborations.
##### Q. Can Duckietown be used for workforce training?
Yes.
Duckietown supports hands-on workforce training in robotics, AI, autonomy, and embodied systems through practical learning experiences combining simulation, software, and physical robots.
Training programs can support engineers, researchers, students, and technical teams working with robotics and AI technologies.
##### Q. Are enterprise partnerships available?
Yes.
Duckietown collaborates with universities, research laboratories, companies, and organizations on robotics and AI initiatives, workforce training, research deployments, grant writing, and technology experimentation.
Partnership structures may include deployment support, customized hardware and software arrangements, collaborative projects, and long-term licensing agreements.
##### Q. Can Duckietown staff run our workforce training courses?
In some cases, yes.
Duckietown primarily provides the robotics platform, software infrastructure, training materials, and deployment support required for hands-on robotics and AI workforce training programs.
Direct involvement from Duckietown staff in delivering workshops, onboarding sessions, or customized training activities may be available for selected programs, institutional deployments, or enterprise collaborations depending on scope, scheduling, and project requirements.
##### Q. Can Duckietown support AI and reinforcement learning (RL) experimentation?
Yes.
Duckietown provides simulation environments, programmable robotics platforms, and modular software tools suitable for reinforcement learning, autonomy development, computer vision, and AI experimentation workflows.
The platform supports both educational and advanced research use cases.
##### Q. Can Duckietown be used for embodied AI research?
Yes.
Duckietown supports embodied AI experimentation through physical robots, simulation environments, and real-world autonomy workflows.
Researchers can experiment with perception, planning, control, navigation, reinforcement learning, and multi-agent robotics systems using both virtual and physical platforms.
### Beginner
experience
Try out Duckietown in simulation
$ 0 Free!
- Duckietown Shell
- Terminal UI
- Duckietown Simulator
- Massive open online course (simulation track)
- Assignment Evaluations
- Duckietown Library
- Community of learners
- Q&A knowledge base
- Community-based support
[ Create account ](https://hub.duckietown.com/signup/) Hardware not included, No credit card needed!
### Roboticist
experience
Learn with a real Duckie-robot
$ 299+
- Same as Beginner experience
- Massive open online course
(sim and hw tracks)
- Powered by Raspberry Pi and NVIDA Jetson Nano
- Browser-based Dashboard
- Open software (white box)
- Modularity: grow setup as you learn
- Self-Driving Cars (Duckiebots)
- Smart-city (Duckietown)
- Quadcopters (Duckiedrones)
- 6 month hardware warranty
- 30 days return no questions
- Worldwide shipping
[ Get hardware ](https://get.duckietown.com/) Starter kit bundles available
---
### [Pricing for universities and schools](https://duckietown.com/pricing/universities-and-schools/)
**Published:** July 17, 2024
**Author:** Duckietown Admin
**Content:**
# Pricing for universities and schools
###### Institutional use of Duckietown in courses, laboratories, and academic programs requires software licenses.
## When is a software license required?
A Duckietown software license is required when the platform is used in:
- university or school courses
- academic teaching or laboratory activities
- institutional research programs
- organized academic or training programs
- student projects that count towards academic requirements, such as theses, capstones, etc.
Duckietown software licenses support maintained software releases, deployment infrastructure, educational resources and tooling, and institutional-scale robotics virtual learning environments.
Software licenses are not required for [independent personal learning](https://duckietown.com/pricing/plans-for-learners/ "Duckietown independent learner license requirements") outside institutional activities.
For research activities, check out the [research and enterprise plans](https://duckietown.com/pricing/plans-for-researchers-and-enterprise/ "Duckietown plans for researchers and enterprise").
### Software
User license required for every learner
- Duckietown Shell (CLI)
- Duckietown desktop applications (GUI)
- Duckiematrix robotics simulator
- Virtual robots
- Massive Open Online course ([MOOC](https://duckietown.com/self-driving-cars-with-duckietown-mooc/))
- Duckietown library
- Community support channels
- Cross-platform support
[ Get a quote ](https://duckietown.com/request-quote/) Bundles and multi-year plans available. Redeem codes when needed.
Hardware sold separately.
### Instructor Resources
Optional exclusive resources for instructors
- Onboarding for teaching staff
- Instructor priority support
- Private support Channel
- Source files to class materials
- Class channel in Duckietown community
- Learning experience deployment support
- Priority Fulfillment
- Early access to new educational materials
[ Get a quote ](https://duckietown.com/request-quote/)
### Hardware
Robots and kits available worldwide
$ 399+
- Powered by Raspberry Pi and NVIDIA Jetson platforms
- Browser-based robotics dashboard
- Fully open software platform
- Modular hardware system that grows with you
- Self-driving cars
- Smart-city robotics environment
- Autonomous quadcopter drones
- 6-month hardware warranty
- 30-day return policy
- Worldwide shipping
[ Get robots and kits ](https://get.duckietown.com/) Classroom bundles available, optional pre-assembly and initialization of Duckiebots for extra time savings.
## Classroom Bundles: quickstart
CLASSROOM KITS
(Hardware ONLY)
class-in-a-box
(All INCLUSIVE)
### Classroom Kits: hardware bundles
Classroom Kits are bundles of [Duckiebots](https://get.duckietown.com/products/duckiebot-db21 "Duckiebot") and city components designed to simplify the choice of hardware for classes or labs.
Variants available:
- different sizes (5, 12, and 30 robots), with city sizes and complexity scaling accordingly
- with or without Jetson Nano compute modules
- DIY assembly, or pre-assembled and initialized
Custom Classroom kit sizes available upon request ([submit a quote request](https://duckietown.com/request-quote/ "Duckietown Request a Quote form")).
[ DIY Classroom Kits ](https://get.duckietown.com/collections/classroom-kits)
### Classroom Kit (5)
Best for informal learning groups
- 5x Duckiebots
- City size: ~10 sqm (100 sqft)
- 1x Traffic Light
- 6 months hardware warranty
[ Request information ](https://duckietown.com/request-quote/)
### Classroom Kit (12)
All to get started with a small class
- 12x Duckiebots
- City size: ~17 sqm (180 sqft)
- 2x Traffic Lights
- 6 months hardware warranty
[ Request information ](https://duckietown.com/request-quote/)
### Classroom Kit (30)
Typical solution for graduate level class
- 30x Duckiebots
- City size: ~22 sqm (240 sqft)
- 3x Traffic Lights
- 6 months hardware warranty
[ Request information ](https://duckietown.com/request-quote/)
- We accept purchase orders: [reach out for exact pricing and a quote](https://duckietown.com/request-quote/ "Duckietown - request a quote form")
- Tailored solutions available: customize your fleet size and city complexity
- Pre-assembly, software initialization, and name customization of Duckiebots is possible
- [Duckiedrone](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24 "Duckiedrone")-based classroom kits available upon request
### Class-in-a-box deployment solution: hardware, software and support
Classes-in-a-box are designed as a “one-click” solution for deploying Duckietown at your institution, bundling all necessary components at a discount.
A Class-in-a-box includes:
- Same hardware as in the Classroom Kits
- Yearly [student software licenses](#university-plans)
- Yearly [instructor pack license](#university-plans)
Discounted multi-year license bundles are available, as well as customized sizes. [Reach out for information and quotes](https://duckietown.com/request-quote/ "Duckietown request a quote page").
[ Class-in-a-box ](https://get.duckietown.com/collections/duckietown-class-in-a-box-collection)
### Frequently asked questions (FAQ)
##### Q. When is a software license required?
A user software license is required when Duckietown is used in courses, laboratories, institutional research, or organized academic programs.
##### Q. What support is available to instructors?
Instructor licenses include access to additional educational support resources, including onboarding assistance, technical and pedagogical guidance, and private support channels.
##### Q. What support is available to students?
Students can access Duckietown community support channels and course resources as part of their learning experience.
Additional institutional support may also be provided by instructors, teaching assistants, or the institution itself.
##### Q. What happens if a software user license is not renewed?
The user will lose access to Duckietown’s latest software release (“ente”) and related resources, but maintain access to Duckietown’s legacy (“daffy”) software release and related resources.
##### Q. Do you accept purchase orders?
Yes.
Duckietown supports institutional procurement workflows, including purchase orders and invoicing for universities, schools, and research organizations.
##### Q. Can licenses be purchased on a multi-year basis?
Yes.
Multi-year licensing arrangements may be available for institutions requiring long-term classroom, laboratory, or research deployments.
##### Q. Do you support international procurement and customs processes?
Yes.
Duckietown ships internationally and can work with institutional procurement departments on invoicing, shipping documentation, customs clearance, and import-related processes.
Requirements may vary depending on the destination country and institution.
##### Q. Is hardware necessary to get started?
No.
Duckietown software can be used without hardware, using the Duckiematrix virtual environment and virtual robots.
Hardware can be introduced later as part of classroom, laboratory, or research activities.
##### Q. Do you offer customized curricula or onboarding support?
Duckietown can support institutions with onboarding, curriculum integration, and deployment guidance depending on the educational program and licensing arrangement. [Reach out](https://duckietown.com/request-quote/) for additional information.
##### Q. Do instructors and students require separate licenses?
User licenses provide access to Duckietown software for coursework and learning activities.
Instructor licenses are optional and provide additional teaching, onboarding, and support resources.
##### Q. Can Duckietown be used in research laboratories?
Yes.
Duckietown supports robotics and AI research activities in universities and research institutions. Here are some examples of [research work produced with Duckietown](https://duckietown.com/research-papers/ "Research with Duckietown").
Research use may require institutional software licenses depending on the deployment model and organization structure.
##### Q. Is Duckietown software open source?
Duckietown includes open-source software components together with maintained educational infrastructure, deployment tooling, documentation, and institutional support resources.
Software licenses support institutional deployment, maintenance, and educational operations.
##### Q. May we rent the hardware instead of purchasing it?
Yes, we currently offer Duckiebot rentals to educational institutions in Europe.
If you are interested in learning more, [reach out to us](https://duckietown.com/request-quote/ "request a quote").
### Class-in-a-box: all inclusive solution for teaching autonomy
### Class-in-a-box (5)
For small groups or shared robots
- All included in Classroom Kit (5)
- Instructor priority support
- Dedicated class channel
- Access to instructor community
- Priority fulfillment
- Shipping insurance
- Early access to new materials
[ Reach Out ](https://duckietown.com/?page_id=99682)
### Class-in-a-box (12)
All to get started with a small class
- All included in Classroom Kit (12)
- 12% discount on hardware
- Student discount for 12x additional robots
- Instructor priority support
- Dedicated class channel
- Access to instructor community
- Priority fulfillment
- Shipping insurance
- Early access to new materials
[ Reach Out ](https://duckietown.com/?page_id=99682)
### Class-in-a-box (30)
Typical setup for university classes
- All included in Classroom Kit (30)
- 15% discount on hardware
- Student discount for 30x additional robots
- Instructor priority support
- Dedicated class channel
- Access to instructor community
- Priority fulfillment
- Shipping insurance
- Early access to new materials
[ Reach Out ](https://duckietown.com/?page_id=99682)
- Tailored solutions possible: customize your fleet size, city complexity, number of teaching assistants, number of classes / year
- Duckiedrone-based classroom kits available upon request
### Class-in-a-box (5)
Starter kit for informal learning groups
$2739
$ 2329 +$159/month (yearly billing)
- 5x Duckiebots
- City size: ~10 sqm (100 sqft)
- Professor experience subscription (1 seat)
- All included in "DIY classroom experience"
- 15% discount on hardware
- 20% discount code for 5x additional student robots
($600 value)
[ Get Class-in-a-box (5) ](https://duckietown.myshopify.com/a/bundles/class-in-a-box-(5)-robotics-club-6gqp) Semester billing: $199/month
### Class-in-a-box (12)
All to get started with a small class
$5999
$ 4999 +$239/month (yearly billing)
- 12x Duckiebots
- City size: ~17 sqm (180 sqft)
- Professor experience subscription (2 seats)
- All included in "DIY classroom experience"
- 17% discount on hardware
- 20% discount code for 12x additional student robots
($1440 value)
[ Get Class-in-a-box (12) ](https://duckietown.myshopify.com/a/bundles/class-in-a-box-(12)-with-jetson-4gb-5yyr) Semester billing: $299/month
Popular
### Class-in-a-box (30)
Typical setup for graduate level class
$13999
$ 11199 +$399/month (yearly billing)
- 30x Duckiebots
- City size: ~22 sqm (240 sqft)
- Professor experience subscription (5 seats)
- All included in "DIY classroom experience"
- 20% discount on hardware
- 20% discount code for 30x additional student robots
($3600 value)
[ Get Class-in-a-box (30) ](https://duckietown.myshopify.com/a/bundles/class-in-a-box-(30)-with-jetson-4gb-5yyt) Semester billing: $499/month
BEST VALUE
### Custom solution
Tell us about your class
Ask
any time
- Support for your teaching staff
- Modular city of customizable size and complexity
- Duckiebots, Duckiedrones or both!
- Volume discount
[ Request a quote ](https://contact.duckietown.com/request-a-quote)
---
### [Duckietown software licenses pricing](https://duckietown.com/pricing/)
**Published:** May 9, 2023
**Author:** Duckietown Admin
**Content:**
# Duckietown Pricing
###### Duckietown software licenses pricing depends on the intended use.
[ I want to learn ](/pricing/independent-learners/)
[ i want to teach ](/pricing/universities-and-schools/)
[ i want to do research ](/pricing/researchers-and-enterprise/)
##### Other uses of Duckietown
For any use of Duckietown other than personal learning, teaching, or research, [reach out](https://duckietown.com/request-quote/) for information.
---
### [Duckietown software license for independent learners](https://duckietown.com/pricing/independent-learners/)
**Published:** July 17, 2024
**Author:** Duckietown Admin
**Content:**
# Duckietown for independent learners
###### Duckietown software is free for personal and non-commercial use. Learn robotics and AI using the same tools used in universities and research labs. No hardware required to get started.
## Duckietown for independent learners
We provide full access to our robotics software, learning experiences, and community support at **no cost** for independent learners using the platform for **personal and non-commercial purposes**.
All that is **required to begin** is a computer and an internet connection.
Duckietown robot kits are optional and available separately through the Duckietown store. Working with hardware is recommended to maximize learning.
If you are an instructor, check out the [academic plans for students and instructors](https://duckietown.com/pricing/plans-for-academia/ "Duckietown academic plans for students and instructors").
If you are a researcher, check out the [research and enterprise plans](https://duckietown.com/pricing/plans-for-researchers-and-enterprise/ "Duckietown plans for researchers and enterprise").
### Software
Free for personal and non-commercial use
$ 0
- Duckietown Shell (CLI)
- Duckietown desktop applications (GUI)
- Duckiematrix robotics simulator
- Virtual robots
- Self-Driving Cars with Duckietown online course
- Duckietown library
- Community support channels
- Cross-platform support (Win, macOS, Ubuntu)
[ Get Started with Duckietown ](https://hub.duckietown.com/signup/) No credit card required. Hardware sold separately.
### Hardware
Robot and kits available worldwide
$ 399+
- Powered by Raspberry Pi and NVIDIA Jetson
- Browser-based robotics dashboard
- Fully open software platform
- Modular hardware system that grows with you
- Self-driving cars
- Smart-city robotics environment
- Autonomous quadcopter drones
- 6-month hardware warranty
- 30-day return policy
- Worldwide shipping
[ Get robots and kits ](https://get.duckietown.com/) Starter kit bundles available.
### Who is the Duckietown independent learner plan for?
###### **Included uses**
The Independent Learner plan is intended for personal, self-directed learning.
Examples include:
- enrolling in the [Self-Driving Cars with Duckietown online course](https://duckietown.com/self-driving-cars-with-duckietown-mooc/ "Self-Driving Cars with Duckietown") for personal interest
- experimenting with robotics and AI at home
- building personal projects
- learning robotics skills independently
###### **Not included**
The Independent Learner plan does not cover institutional or commercial use.
Examples include:
- use within a school or university course
- academic teaching or laboratory activities
- research projects conducted by an organization
- commercial or company use
### Frequently asked questions (FAQ)
##### Q. Is Duckietown really free?
Yes.
Duckietown software is free for personal and non-commercial use under the Independent Learner plan.
No payment or credit card is required to create an account.
##### Q. Do I need to buy a robot to start?
No.
Duckietown software can be used without hardware, using simulation and virtual robots ([check the Duckiematrix documentation](https://docs.duckietown.com/ente/duckietown-manual/50-duckiematrix/introduction-to-the-duckiematrix-virtual-environment.html "Duckietown Simulation and the Duckiematrix")).
Of course, the nuisances of robotics are best experienced through hands-on work with hardware, but this is optional and can be added later.
##### Q. What counts as personal and non-commercial use?
Personal use means learning or experimenting independently.
Examples include:
- taking the Self-Driving Cars with Duckietown course
- building robotics projects at home
- learning robotics skills independently
##### Q. When do I need a paid license?
You need a paid license if Duckietown is used in:
- a school or university course
- an academic research lab
- a company or commercial environment
- an organized training program
##### Q. How do I get access to Duckietown software?
Create a Duckietown account and select the Independent Learner plan.
Access is activated immediately after account creation. You can create an account through the [Duckietown Hub signup page](https://hub.duckietown.com/signup/ "Signup to Duckietown").
##### Q. Can I upgrade my plan later?
Yes.
You can upgrade to an academic or professional license at any time if your use changes.
##### Q. What kind of computer do I need?
A mid-tier commercial laptop is sufficient to work with Duckietown. More detailed specifications are provided on the [Duckietown Documentation – Your Computer](https://docs.duckietown.com/ente/duckietown-manual/10-setup/00-computer/installing-ubuntu-on-your-computer.html#computer-and-internet-requirements-for-using-duckietown "Duckietown computer minimum and required specifications") page.
##### Q. What kind of internet connection do I need to learn with Duckietown?
A non-metered broadband internet connection. You will have to download tens of gigabytes of data (learning experiences, software updates, etc.).
##### Q. Do I need programming experience?
Some experience in any programming language will be helpful.
Duckietown provides guided learning experiences for beginners as well as advanced robotics tools for experienced users.
##### Q. What is the Duckietown Independent Learner plan?
The Independent Learner plan provides free access to Duckietown robotics software for personal and non-commercial use.
### Beginner
experience
Try out Duckietown in simulation
$ 0 Free!
- Duckietown Shell
- Terminal UI
- Duckietown Simulator
- Massive open online course (simulation track)
- Assignment Evaluations
- Duckietown Library
- Community of learners
- Q&A knowledge base
- Community-based support
[ Create account ](https://hub.duckietown.com/signup/) Hardware not included, No credit card needed!
### Roboticist
experience
Learn with a real Duckie-robot
$ 299+
- Same as Beginner experience
- Massive open online course
(sim and hw tracks)
- Powered by Raspberry Pi and NVIDA Jetson Nano
- Browser-based Dashboard
- Open software (white box)
- Modularity: grow setup as you learn
- Self-Driving Cars (Duckiebots)
- Smart-city (Duckietown)
- Quadcopters (Duckiedrones)
- 6 month hardware warranty
- 30 days return no questions
- Worldwide shipping
[ Get hardware ](https://get.duckietown.com/) Starter kit bundles available
---
### [Get a Duckietown quote](https://duckietown.com/request-quote/)
**Published:** December 2, 2023
**Author:** Duckietown Admin
**Content:**
# Get a quote
###### Reach out to request a quote for Duckietown products and services.
Having trouble viewing this page? Go to [https://contact.duckietown.com/request-a-quote](https://contact.duckietown.com/request-a-quote "Duckietown request a quote") to submit your quote request.
---
### [Self-Driving Cars with Duckietown: learn the technology of autonomous vehicles](https://duckietown.com/self-driving-cars-with-duckietown-mooc/)
**Published:** October 4, 2020
**Author:** Duckietown Admin
**Content:**
# Self-Driving Cars with Duckietown
###### The world’s first robot autonomy massive open online course with hardware
[ Enroll now ](#mooc-quickstart-links)
## Self-Driving Cars with Duckietown Massive Open Online Course (MOOC)
Self-Driving Cars with Duckietown is the world’s first **hardware-based massive open online course (MOOC) in AI robotics**.
Created in collaboration with ETH Zürich, the University of Montreal, and the Toyota Technological Institute at Chicago (TTIC), Self-Driving Cars with Duckietown is **free** to enroll in.
Hosted on the edX platform, Self-Driving Cars with Duckietown is a “grand tour” of robot autonomy: the science and technology behind enabling machines to make their own decisions and accomplish broadly defined tasks such as lane following, object detection, planning, and more.
Self-driving cars with Duckietown covers from the **theory**, to the **implementation**, to the **deployment**: in **simulation**, as well as on Duckiebots (**real-world model autonomous vehicles**).
Starting from the 2025 edition, Self-Driving Cars with Duckietown now supports virtual Duckiebots: digital twins of physical robots, for an even more accessible entry to the beautiful world of physical AI.
[ Enroll Now (2025 ente edition) ](https://www.edx.org/learn/technology/eth-zurich-self-driving-cars-with-duckietown)
[ Get a Duckiebot Starter Kit ](https://cutt.ly/website-mooc-hw-starter-kit-3x3)
[  ](https://get.duckietown.com/)
"I’m thrilled that ETH, with UMontreal, the Duckietown Foundation, and the Toyota Technological Institute in Chicago, are collaborating to bring this course in self-driving cars and robotics to the 35 million learners on edX. This emerging technology has the potential to completely change the way we live and travel, and the course provides a unique opportunity to get in on the ground floor of understanding and using the technology powering autonomous vehicles."
Anant AgarwalFounder and CEO of edX, Professor at the Massachussetts Institute of Technology (MIT)
"The new NVIDIA Jetson Nano is the ultimate starter AI computer for educators and students to teach and learn AI at an incredibly affordable price. Duckietown and its EdX MOOC are leveraging Jetson to take hands-on experimentation and understanding of AI and autonomous machines to the next level."
Deepu TallaVice President and General Manager of Edge Computing at NVIDIA
"The Duckietown educational platform provides a hands-on, scaled down, accessible version of real world autonomous systems. Integrating NVIDIA’s Jetson Nano power in Duckietown enables unprecedented access to state-of-the-art compute solutions for learning autonomy."
Emilio FrazzoliProfessor at the Swiss Federal Institute of Technology in Zurich (ETHZ)
### MOOC Factsheet
- Name: Self-driving cars with Duckietown
- Platform: edX
- Cost: free to enroll
- Instructors: Swiss Federal Institute of Technology in Zurich (ETHZ), Université de Montréal (UdM), Toyota Technological Institute at Chicago (TTIC)
### Prerequisites
- Basic Linux, Python, Git
- Elements of linear algebra, probability, calculus
- Elements of kinematics, dynamics
- Computer with native Ubuntu installation
- Broadband internet connection
### What you will learn
- Computer Vision
- Robot operations
- Differential drive Modeling
- Object Detection
- Onboard localization
- Robot Control (PID)
- Planning
- Reinforcement Learning
[ Preview the lectures ](https://www.youtube.com/playlist?list=PLA_e-PnzHulcMKknwJCtqEzP-n4iEjE8C)
[ Enroll Now (2025 ente edition) ](https://www.edx.org/learn/technology/eth-zurich-self-driving-cars-with-duckietown)
([Archived Self-Driving Cars with Duckietown (daffy edition) 2023-24](https://learning.edx.org/course/course-v1:ETHx+DT-01x+3T2023/home "Self-Driving Cars with Duckietown (daffy edition) 2023-24"))
**Robot Perception**: Duckiebots detect lane marking on the fly and use mathematical models of their camera and environment to estimate their position and orientation in the lane.
**Duckiebot Detection**: driving in Duckietown is fun but safety should always be paramount. DuckieBots can detect other vehicles and estimate their relative poses to avoid collisions.
**Robot Planning**: as Duckietowns grow bigger, smart Duckiebots plan their path in town. Traffic signs at intersections provide landmarks to localize on the global map and determine next turns.
**Pedestrian detection**: there are many obstacles in Duckietown – some move and some don’t. Being able to detect pedestrians (duckies) is important to guarantee safe driving.
"Autonomous driving is one of the most active AI fields, and this course uniquely tackles online learning for self-driving vehicles by pairing its content with a purchasable driving robot. The course itself digs deep into controlling your Duckiebot, such as how to drive in lanes, stop at intersections, and detect and avoid crashing into objects."
![LearnDataSci]()
LearnDataSci**4,9/5**
"This excellent edX course from ETH Zurich will guide you through the process of programming and building self-driving cars using the Duckietown Robotic Ecosystem, an open-source platform created by MIT’s CSAIL. Unlike all other courses on the list, this one will teach you to build a basic but actual self-driving car."
![Victory Tale]()
Victory Tale**4,8/5**
"If you want to learn about autonomous vehicles and how they make a significant impact on society, this course from edX is an excellent option for you. Taking part in this program will take you on a learning journey of planning and developing self-driving vehicles. You will begin learning from a box of components used in auto-creation and then move on to create your scaled self-driving car that drives autonomously in your living room."
![DigitalDefynd]()
DigitalDefynd**4,4/5**
[ Enroll now ](#mooc-quickstart-links)
## Self-Driving Cars with Duckietown Massive Open Online Course (MOOC)
Self-Driving Cars with Duckietown is the world’s first **hardware-based massive open online course (MOOC) in AI robotics**.
Created in collaboration with ETH Zürich, the University of Montreal, and the Toyota Technological Institute at Chicago (TTIC), Self-Driving Cars with Duckietown is **free** to enroll in.
Hosted on the edX platform, Self-Driving Cars with Duckietown is a “grand tour” of robot autonomy: the science and technology behind enabling machines to make their own decisions and accomplish broadly defined tasks such as lane following, object detection, planning, and more.
Self-driving cars with Duckietown covers from the **theory**, to the **implementation**, to the **deployment**: in **simulation**, as well as on Duckiebots (**real-world model autonomous vehicles**).
Starting from the 2025 edition, Self-Driving Cars with Duckietown now supports virtual Duckiebots: digital twins of physical robots, for an even more accessible entry to the beautiful world of physical AI.
[ Enroll Now (2025 ente edition) ](https://www.edx.org/learn/technology/eth-zurich-self-driving-cars-with-duckietown)
[ Get a Duckiebot Starter Kit ](https://cutt.ly/website-mooc-hw-starter-kit-3x3)
[  ](https://get.duckietown.com/)
"I’m thrilled that ETH, with UMontreal, the Duckietown Foundation, and the Toyota Technological Institute in Chicago, are collaborating to bring this course in self-driving cars and robotics to the 35 million learners on edX. This emerging technology has the potential to completely change the way we live and travel, and the course provides a unique opportunity to get in on the ground floor of understanding and using the technology powering autonomous vehicles."
Anant AgarwalFounder and CEO of edX, Professor at the Massachussetts Institute of Technology (MIT)
"The new NVIDIA Jetson Nano is the ultimate starter AI computer for educators and students to teach and learn AI at an incredibly affordable price. Duckietown and its EdX MOOC are leveraging Jetson to take hands-on experimentation and understanding of AI and autonomous machines to the next level."
Deepu TallaVice President and General Manager of Edge Computing at NVIDIA
"The Duckietown educational platform provides a hands-on, scaled down, accessible version of real world autonomous systems. Integrating NVIDIA’s Jetson Nano power in Duckietown enables unprecedented access to state-of-the-art compute solutions for learning autonomy."
Emilio FrazzoliProfessor at the Swiss Federal Institute of Technology in Zurich (ETHZ)
### MOOC Factsheet
- Name: Self-driving cars with Duckietown
- Platform: edX
- Cost: free to enroll
- Instructors: Swiss Federal Institute of Technology in Zurich (ETHZ), Université de Montréal (UdM), Toyota Technological Institute at Chicago (TTIC)
### Prerequisites
- Basic Linux, Python, Git
- Elements of linear algebra, probability, calculus
- Elements of kinematics, dynamics
- Computer with native Ubuntu installation
- Broadband internet connection
### What you will learn
- Computer Vision
- Robot operations
- Differential drive Modeling
- Object Detection
- Onboard localization
- Robot Control (PID)
- Planning
- Reinforcement Learning
[ Preview the lectures ](https://www.youtube.com/playlist?list=PLA_e-PnzHulcMKknwJCtqEzP-n4iEjE8C)
[ Enroll Now (2025 ente edition) ](https://www.edx.org/learn/technology/eth-zurich-self-driving-cars-with-duckietown)
([Archived Self-Driving Cars with Duckietown (daffy edition) 2023-24](https://learning.edx.org/course/course-v1:ETHx+DT-01x+3T2023/home "Self-Driving Cars with Duckietown (daffy edition) 2023-24"))
**Robot Perception**: Duckiebots detect lane marking on the fly and use mathematical models of their camera and environment to estimate their position and orientation in the lane.
**Duckiebot Detection**: driving in Duckietown is fun but safety should always be paramount. DuckieBots can detect other vehicles and estimate their relative poses to avoid collisions.
**Robot Planning**: as Duckietowns grow bigger, smart Duckiebots plan their path in town. Traffic signs at intersections provide landmarks to localize on the global map and determine next turns.
**Pedestrian detection**: there are many obstacles in Duckietown – some move and some don’t. Being able to detect pedestrians (duckies) is important to guarantee safe driving.
"Autonomous driving is one of the most active AI fields, and this course uniquely tackles online learning for self-driving vehicles by pairing its content with a purchasable driving robot. The course itself digs deep into controlling your Duckiebot, such as how to drive in lanes, stop at intersections, and detect and avoid crashing into objects."
![LearnDataSci]()
LearnDataSci**4,9/5**
"This excellent edX course from ETH Zurich will guide you through the process of programming and building self-driving cars using the Duckietown Robotic Ecosystem, an open-source platform created by MIT’s CSAIL. Unlike all other courses on the list, this one will teach you to build a basic but actual self-driving car."
![Victory Tale]()
Victory Tale**4,8/5**
"If you want to learn about autonomous vehicles and how they make a significant impact on society, this course from edX is an excellent option for you. Taking part in this program will take you on a learning journey of planning and developing self-driving vehicles. You will begin learning from a box of components used in auto-creation and then move on to create your scaled self-driving car that drives autonomously in your living room."
![DigitalDefynd]()
DigitalDefynd**4,4/5**
---
### [Educational resources: teach and learn robotics and AI from here](https://duckietown.com/educational-resources/)
**Published:** December 27, 2022
**Author:** Duckietown Admin
**Content:**
# Educational resources
###### We offer university-level classroom resources for teaching and learning robot autonomy. These materials have been used at 300+ universities worldwide.
## Community Resources
The Duckietown curriculum is structured in “modules”, each of which is supported by several types of materials to reinforce it.
All modules have slides. In the “Extra Materials” column below:
 are links to **notes**
 are links to **additional videos**
 are links to **short videos** (MOOC format)
 are links to **learning activities** to be run on in simulation and/or a Duckiebot
If you have resources you would like to share with the community, [reach out to us](/contact)!
### Examples
Want ideas on how to structure your class?
- Université de Montréal: [Autonomous Vehicles, 2025 class](https://liampaull.ca/ift6757/ "Autonomous Vehicles with Duckietown course, 2025 class")
- ETH Zürich: [Autonomous Mobility on Demand, 2020 class](https://idsc.ethz.ch/education/lectures/duckietown.html "ETH Zurich Duckietown class: Autonomous Mobility on Demand, 2020")
- UMass Lowell: [Fundamentals of Robotics, 2021](https://sites.uml.edu/paul-robinette/teaching/ "University of Massachusetts Lowell Duckietown class 2021 - Fundamentals of Robotics")
#### Autonomy basics and traditional approaches
##### Introduction
###### Introduction - Duckietown
- Duckietown project \[[pdf](https://hubs.ly/Q02b8k1P0)\]
- Duckietown Platform \[[pdf](https://hubs.ly/Q02b8kfv0)\]
- AI Driving Olympics \[[pdf](https://hubs.ly/Q02b8kpP0)\]
[1-MIT2016-1](https://vimeo.com/784311587), [1-TTIC2017](https://vimeo.com/240901361 "TTIC lesson 1 - Introduction to Duckietown")[1-Intro to AVs](https://vimeo.com/duckietown/01-intro-to-avs)[Laptop setup](https://docs.duckietown.com/ente/duckietown-manual/10-setup/00-computer/installing-ubuntu-on-your-computer.html "Duckietown Laptop Setup Instructions"), [Accounts](https://docs.duckietown.com/ente/duckietown-manual/10-setup/01-accounts/accounts-introduction.html "Duckietown guide to accounts creation")
###### Introduction - Autonomous Vehicles
- Intro to AVs \[[pdf](https://hubs.ly/Q02b8ktp0)\]
[1-MIT2016-2](https://vimeo.com/784313289), [5-MIT2016-3](https://vimeo.com/784458132)[2- Auto](https://vimeo.com/532398159), [3-Levels](https://vimeo.com/535264137), [4-Vision](https://vimeo.com/538089904), [5-Robots](https://vimeo.com/542137594)[Software Environment Setup](https://docs.duckietown.com/ente/duckietown-manual/10-setup/02-software/sw-introduction.html "Software Environment Setup"), [Duckiematrix Installation](https://docs.duckietown.com/ente/duckietown-manual/50-duckiematrix/introduction-to-the-duckiematrix-virtual-environment.html "Duckiematrix Installation")
###### Introduction - Robotic Systems
- Modern Robotic Systems \[[pdf](https://hubs.ly/Q02b8kxG0)\]
- Architectures \[[pdf](https://hubs.ly/Q02b8kCV0)\]
- Testing – part 1 \[[pdf](https://hubs.ly/Q02b8kP10)\]
[3-TTIC2017](https://vimeo.com/240901460), [7-1-MIT2016](https://vimeo.com/798727385)[5-Robots](https://vimeo.com/542137594), [6-BVs](https://vimeo.com/546397378)[LX-Braitenberg Vehicles](https://docs.duckietown.com/ente/duckietown-manual/60-learning-experiences/lx-setup-braitenberg-vehicles.html "LX-Braitenberg Vehicles (ente)"), [Duckiebot Assembly](https://docs.duckietown.com/ente/duckietown-manual/10-setup/03-duckiebot/assembly-instructions-duckiebot-db21j.html "Duckiebot DB21 Assembly Instructions"), [Duckietown Assembly](https://docs.duckietown.com/ente/duckietown-manual/30-duckietown-city/duckietowns-intro.html "Duckietown Assembly Instructions")
###### Introduction - Software
- Testing – part 2 \[[pdf](https://hubs.ly/Q02b8ll50)\]
- Version control \[[pdf](https://hubs.ly/Q02b8lm_0)\]
- Containerization \[[pdf](https://hubs.ly/Q02b8lXg0)\]
- Networking \[[pdf](https://hubs.ly/Q02b8m1x0)\]
- Modern Signal Processing \[[pdf](https://hubs.ly/Q02b8m230)\]
[7-2-MIT2016](https://vimeo.com/798727385)[Git](https://vimeo.com/526923344), [Docker](https://vimeo.com/527006910)[Duckiebot Initialization](https://docs.duckietown.com/ente/duckietown-manual/10-setup/03-duckiebot/flashing-sd-card-duckiebot-initialization.html "Duckiebot Initialization")
###### Middleware Architectures
- Middleware and ROS \[[pdf](https://hubs.ly/Q02b8m3J0)\]
- Autonomy Architectures \[[pdf](https://hubs.ly/Q02b8m5h0)\]
[4-MIT2016-1](https://vimeo.com/784320705), [4-MIT2016-2](https://vimeo.com/784342597)[7-Stateful](https://vimeo.com/554223650), [8-Logical](https://vimeo.com/587453397)[LX-ROS Basics](https://docs.duckietown.com/ente/duckietown-manual/60-learning-experiences/lx-setup-ros-basics.html "Duckietown Introduction to ROS")
##### Modeling, Kinematics, and Dynamics
###### Modeling and Control
- Representations – part 1 \[[pdf](https://hubs.ly/Q02bk6-V0)\]
- Modeling \[[pdf](https://hubs.ly/Q02bk6_l0)\]
- Odometry Calibration \[[pdf](https://hubs.ly/Q02bk6_K0)\]
- Intro to Control Systems \[[pdf](https://hubs.ly/Q02bk6_T0)\]
- Control in Duckietown \[[pdf](https://hubs.ly/Q02bk70t0)\]
[4-TTIC2017](https://vimeo.com/240901462), [5-TTIC2017](https://vimeo.com/240901466)[9-Control](https://vimeo.com/587970827), [10-Representations](https://vimeo.com/588472832), [11-Modeling](https://vimeo.com/587974152), [12-Odometry](https://vimeo.com/580764763), [13-PID](https://vimeo.com/588460419)[LX-Kinematics and Odometry](https://docs.duckietown.com/ente/duckietown-manual/60-learning-experiences/lx-setup-modeling-kinematics.html "LX-Kinematics and Odometry"), [LX-PID Control](https://docs.duckietown.com/ente/duckietown-manual/60-learning-experiences/lx-setup-pid-control.html "PID Control Learning Experience")
##### Computer Vision
###### Principles of Vision
- Computer Vision: Overview \[[pdf](https://hubs.ly/Q02bk72G0)\]
- Image acquisition \[[pdf](https://hubs.ly/Q02bk7310)\]
- Pinhole Camera model \[[pdf](https://hubs.ly/Q02bk73b0)\]
- Camera Calibration \[[pdf](https://hubs.ly/Q02bk75q0)\]
[MIT2016](https://vimeo.com/784340968), [6-TTIC2017](https://vimeo.com/240901471)[14-Projective](https://vimeo.com/797335629), [15-Calibration](https://vimeo.com/549643385)[Camera Calibration](https://docs.duckietown.com/ente/duckietown-manual/20-operations/04-calibrations/duckiebot-camera-calibration.html#db-camera-calibration "Duckiebot Camera Calibration")
###### Feature Detection
- Robust Fitting \[[pdf](https://hubs.ly/Q02bk77k0)\]
- Image Filtering \[[pdf](https://hubs.ly/Q02bk7gg0)\]
- Image Gradients \[[pdf](https://hubs.ly/Q02bk7xn0)\]
- Edge and Corner Detection \[[pdf](https://hubs.ly/Q02bk7_m0)\]
[7-TTIC2017](https://vimeo.com/240901476), [8-TTIC2017](https://vimeo.com/240901487)[16-Filtering](https://vimeo.com/549643768)[LX-Computer Vision](https://docs.duckietown.com/ente/duckietown-manual/60-learning-experiences/lx-setup-computer-vision.html "LX-Computer Vision")
##### Estimation
###### Filtering
- Representations – part 2 \[[pdf](https://hubs.ly/Q02bk8cz0)\]
- Bayes Filter \[[pdf](https://hubs.ly/Q02bk8h20)\]
- Particle Filter \[[pdf](https://hubs.ly/Q02bk8CJ0)\]
- Lane Filter \[[pdf](https://hubs.ly/Q02bk9b90)\]
[MIT2016](https://vimeo.com/784456701), [10-TTIC2017-2](https://vimeo.com/241971536#t=2389), [11-TTIC2017](https://vimeo.com/241971538)[21-Bayes](https://vimeo.com/588996131), [22-Kalman](https://vimeo.com/889271993 "Kalman filterting"), [23-Particle](https://vimeo.com/589420968), [24-Histogram](https://vimeo.com/589416875)[LX: Localization – Extended Kalman Filter (EKF)](https://docs.duckietown.com/ente/duckietown-manual/60-learning-experiences/lx-setup-ekf-extended-kalman-filter-localization.html "LX: Localization - Extended Kalman Filter (EKF)")
###### RANSAC, Place Recognition
- RANSAC \[[pdf](https://hubs.ly/Q02bk9bX0)\]
- Place Recognition \[[pdf](https://hubs.ly/Q02bk9sD0)\]
[10-TTIC2017-1](https://vimeo.com/241971536)
###### SLAM
- SLAM \[[pdf](https://hubs.ly/Q02bkbcf0)\]
- Advanced SLAM \[[link](https://hubs.ly/Q02bkbcD0)\]
[JL-MIT2016](https://vimeo.com/784456992), [19-MIT2016](https://vimeo.com/784712138), [11-TTIC2017](https://vimeo.com/241971540)
##### Planning
###### Planning
- Graphs \[[pdf](https://hubs.ly/Q02bkbkH0)\]
- Motion Planning \[[pdf](https://hubs.ly/Q02bkbnG0)\]
- Sampling-based \[[pdf](https://hubs.ly/Q02bkbsN0)\]
[15-1-MIT2016](https://vimeo.com/786499947), [15-2MIT2016](https://vimeo.com/786536573), [15-3-MIT2016](https://vimeo.com/786809748)[26-PlanningOptimal](https://vimeo.com/587468356), [27-PathsAndTrajectories](https://vimeo.com/587467120), [28-NonCollision](https://vimeo.com/manage/videos/587465042), [29-Graphs](https://vimeo.com/manage/videos/587465904), [30-AdvGraphs](https://vimeo.com/587470071), [31-Search](https://vimeo.com/587469264)[LX-CollisionChecker](https://github.com/duckietown/duckietown-lx/tree/mooc2022/collision-checker), [LX-Planning](https://github.com/duckietown/duckietown-lx/tree/mooc2022/planning)
#### Advanced autonomy approaches
##### Multi-vehicle
###### Multi-vehicle Coordination
- Coordination \[[pdf](https://hubs.ly/Q02bkbJh0)\]
[18-MIT2016](https://vimeo.com/786841986)
###### Fleet-Level Planning
- Fleet Planning \[[pdf](https://hubs.ly/Q02bkbTP0)\]
###### Autonomous Mobility on Demand
- AMoD \[[pdf](https://hubs.ly/Q02bkbX60)\]
##### Machine Learning
###### ML in Robotics
- ML in Robotics \[[pdf](https://hubs.ly/Q02bkbXm0)\]
[18-NNintro](https://vimeo.com/549674943)
###### Robots perception
- Robotic perception \[[pdf](https://hubs.ly/Q02bkbYR0)\]
[17-AdvancedPerception](https://vimeo.com/549675016), [19-DeepConvNet](https://vimeo.com/549675144), [20-ObjectDetection](https://vimeo.com/549674863)[LX-ObjectDetection](https://docs.duckietown.com/ente/duckietown-manual/60-learning-experiences/lx-setup-ml-based-object-detection.html "LX-ObjectDetection")
###### Reinforcement Learning
- Reinforcement Learning \[[pdf](https://hubs.ly/Q02bkbYj0)\]
[32-MDP](https://vimeo.com/587461397), [33-Policy](https://vimeo.com/587462246), [34-Q](https://vimeo.com/587458627), [35-Sim2Real](https://vimeo.com/587460013)
##### Human-Machine Interaction and Safety
###### Introduction to Safety
- Safety \[[pdf](https://hubs.ly/Q02bkb_60)\]
[17-2-MIT2016](https://vimeo.com/798727385)
###### Advanced Safety and Formal Methods
- Advanced Safety \[[pdf](https://hubs.ly/Q02bkc2j0)\]
[17-MIT2016](https://vimeo.com/798725628)
##### Advanced Perception
###### Estimation from motion blur
- Estimation from motion blur \[[pdf](https://hubs.ly/Q02bkf7n0)\]
###### (Hidden Template)
- Template \[pdf\]
   
## (hidden) Educational Resources
We have structured our curriculum in terms of “modules” where each module is supported by several types of materials to reinforce it. All modules have slides.
In the “Extra Materials” column below:
are links to notes in [the duckiebook](http://docs.duckietown.com/)
are links to exercises in [the duckiebook](http://docs.duckietown.com/)
are links to Jupyter notebooks in [the duckiebook](http://docs.duckietown.com/)
are links to demos to be run on the Duckiebot
are links to additional videos that have been created
Want some ideas about how to structure your class?
Université de Montréal’s Fall 2019 [syllabus](https://liampaull.ca/ift6757/)
ETH Zürich Fall 2019 [class outline](https://idsc.ethz.ch/education/lectures/duckietown.html)
###### Topic
###### Lecture Slides
###### Lecture Recordings
###### Extra Materials
## Autonomy Basics Material
### Introduction
Introduction – Duckietown[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2018/duckietown_intro.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/duckietown_intro.pdf)[From TTIC 2017 (Matt Walter)](https://vimeo.com/240901361)Introduction – Autonomous Vehicles[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2018/autonomous_vehicles.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/autonomous_vehicles.pdf)[Autonomous Vehicles ](http://docs.duckietown.com/DT18/learning_materials/out/autonomous_vehicles.html)Introduction – Autonomy[\[keynote\]](https://github.com/duckietown/lectures/blob/master/1_ideal/05_AV_intro/autonomy_overview.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/1_ideal/05_AV_intro/autonomy_overview.pdf) Introduction – Robotic Systems[\[keynote\]](https://github.com/duckietown/lectures/blob/master/1_ideal/10_architectures/modern_robotic_systems.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/1_ideal/10_architectures/modern_robotic_systems.pdf)[Modern Robotic Systems](http://docs.duckietown.com/DT18/learning_materials/out/modern_robotic_systems.html)
[From TTIC 2017 (Matt Walter)](https://vimeo.com/240901460)Introduction – Systems Architectures[\[keynote\]](https://github.com/duckietown/lectures/blob/master/1_ideal/10_architectures/systems_architecture_basics.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/1_ideal/10_architectures/systems_architecture_basics.pdf)[System architecture basics](http://docs.duckietown.com/DT18/learning_materials/out/system_architectures_basics.html)
[From MIT 2016 (Misha Novitzky)](https://vimeo.com/156041019)Introduction – Autonomy Architectures[\[keynote\]](https://github.com/duckietown/lectures/blob/master/1_ideal/10_architectures/autonomy_architectures.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/1_ideal/10_architectures/autonomy_architectures.pdf)[Autonomy architectures](http://docs.duckietown.com/DT18/learning_materials/out/autonomy_architectures.html)
[From MIT 2016 (Misha Novitzky)](https://vimeo.com/156041019)Introduction – Representations[\[keynote\]](https://github.com/duckietown/lectures/blob/master/1_ideal/25_computer_vision/representations.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/1_ideal/25_computer_vision/representations.pdf)[Representations](http://docs.duckietown.com/DT18/learning_materials/out/representations.html)Tools – Networking[\[keynote\]](https://github.com/duckietown/lectures/blob/master/1_ideal/15_networking/networking.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/1_ideal/15_networking/networking.pdf) Tools – Version Control[\[keynote\]](https://github.com/duckietown/lectures/blob/master/1_ideal/16_version_control/version_control.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/1_ideal/16_version_control/version_control.pdf)[Git and Github](http://docs.duckietown.com/DT18/software_reference/out/ref_git_github.html)Tools – Middlewares (ROS)[\[keynote\]](https://github.com/duckietown/lectures/blob/master/1_ideal/17_ros/introduction_to_ros.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/1_ideal/17_ros/introduction_to_ros.pdf)[ROS installation and reference](http://docs.duckietown.com/DT18/software_reference/out/introduction_to_ros.html)
[Taking and verifying a log](http://docs.duckietown.com/DT18/opmanual_duckiebot/out/take_a_log.html)
[From MIT 2016 (Shih-Yuan Liu)](https://vimeo.com/156041024)### Modeling, Kinematics, and Dynamics
Modeling[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2018/modeling.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/modeling.pdf)[Duckiebot modeling](http://docs.duckietown.com/DT18/learning_materials/out/duckiebot_modeling.html)
[From TTIC 2017 (Matt Walter)](https://vimeo.com/240901466)Calibration – Odometry[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2018/odometry_calibration.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/odometry_calibration.pdf)[Wheel calibration](http://docs.duckietown.com/DT18/opmanual_duckiebot/out/wheel_calibration.html)Signal Processing[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2018/modern_signal_processing.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/modern_signal_processing.pdf)[Modern signal processing](http://docs.duckietown.com/DT18/learning_materials/out/modern_signal_processing.html)### Computer Vision
Basics[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2018/cv_basics.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/cv_basics.pdf)[Basic image operations](http://docs.duckietown.com/DT18/exercises/out/exercise_basic_image.html)
[Instagram filters](http://docs.duckietown.com/DT18/exercises/out/exercise_instagram.html)
[OpenCV basics](https://colab.research.google.com/drive/1RWGmqoEQdeyh5TssoGtsXsFk8hbLGtWp#scrollTo=WtlNW7sC6xhT)
[From TTIC 2017 (Matt Walter)](https://vimeo.com/240901471)
[Augmented Reality](http://docs.duckietown.com/DT18/exercises/out/exercise_augmented_reality.html)
[From TTIC 2017 (Matt Walter)](https://vimeo.com/240901476)Feature Extraction[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2018/feature_extraction.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/feature_extraction.pdf) Line Detection[\[keynote\]](https://github.com/duckietown/lectures/blob/master/1_ideal/25_computer_vision/line_detection.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/1_ideal/25_computer_vision/line_detection.pdf)[From TTIC 2017 (Matt Walter)](https://vimeo.com/240901487)Place Recognition[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2018/place_recognition.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/place_recognition.pdf)[From TTIC 2017 (Matt Walter)](https://vimeo.com/241971536)RANSAC[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2018/ransac.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/ransac.pdf) Camera Calibration[\[keynote\]](https://github.com/duckietown/lectures/blob/master/1_ideal/25_computer_vision/cv_calibration.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/1_ideal/25_computer_vision/cv_calibration.pdf)[Camera calibration and validation](http://docs.duckietown.com/DT18/opmanual_duckiebot/out/camera_calib.html)### Estimation
Bayes Filter[\[pptx\]](https://github.com/duckietown/lectures/blob/master/2018/bayes_filter.pptx)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/bayes_filter.pdf)[Probability basics](http://docs.duckietown.com/DT18/preliminaries/out/probability_basics.html)
[From TTIC 2017 (Matt Walter)](https://vimeo.com/241971538)
[From MIT 2016 (John Leonard)](https://vimeo.com/157886581)
[From MIT 2016 (Liam Paull)](https://vimeo.com/157886586)Particle Filter[\[pptx\]](https://github.com/duckietown/lectures/blob/master/2018/particle_filter.pptx)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/particle_filter.pdf)[Particle Filter](http://docs.duckietown.com/DT18/exercises/out/exercise_filtering_pf.html)Lane Filter[\[pptx\]](https://github.com/duckietown/lectures/blob/master/2018/lane_filter.pptx)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/lane_filter.pdf)[Lane Following](http://docs.duckietown.com/DT18/opmanual_duckiebot/out/demo_lane_following.html)Kalman Filter[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2_given/Forbes_KF.pdf)[Lane Filtering – Extended Kalman Filtering](http://docs.duckietown.com/DT18/exercises/out/exercise_filtering_ekf.html)SLAM[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2018/slam_intro.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/slam_intro.pdf)[From TTIC 2017 (Matt Walter)](https://vimeo.com/241971540)### Planning and Control
Control[\[pptx\]](https://github.com/duckietown/lectures/blob/master/2018/control.pptx)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/control.pdf)[Make Way for Duckiebots](http://docs.duckietown.com/DT18/opmanual_duckiebot/out/demo_follow_leader.html)
[Lane Following](http://docs.duckietown.com/DT18/opmanual_duckiebot/out/demo_lane_following.html)Motion Planning[\[pptx\]](https://github.com/duckietown/lectures/blob/master/2018/motion_planning.pptx)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/motion_planning.pdf)[Indefinite Navigation](http://docs.duckietown.com/DT18/opmanual_duckiebot/out/demo_indefinite_navigation.html)### Testing
Testing, Validation, Verification[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2018/testing.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/pdfs/testing.pdf)[Who watches the watchmen?](http://docs.duckietown.com/DT18/exercises/out/exercise_watchmen.html)## Advanced Autonomy Material
### Multi-Vehicle
Multi-Vehicle Coordination[\[pptx\]](https://github.com/duckietown/lectures/blob/master/3_legacy/C18_multi_robot_coordination/JA-Lecture18-optimCoord.pptx)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/3_legacy/C18_multi_robot_coordination/JA-Lecture18-optimCoord.pdf) Fleet-level Planning[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2_given/2017-11-22-%20ETHZ%20-Duckietown_FleetControl_(Claudio%20Ruch).key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2_given/2017-11-22-%20ETHZ%20-Duckietown_FleetControl_(Claudio%20Ruch).pdf)[Fleet planning](http://docs.duckietown.com/DT18/opmanual_duckiebot/out/demo_fleet_planning.html)Introduction to Autonomous Mobility on Demand[\[keynote\]](https://github.com/duckietown/lectures/blob/master/1_ideal/05_AV_intro/AMOD_intro.key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/1_ideal/05_AV_intro/AMOD_intro.pdf) ### Machine Learning
Machine Learning and Deep Learning – Introduction[\[Google slides\]](https://docs.google.com/presentation/d/1T5jY33_plvfnXqpqgS1lhS_m-iMAEB89VOC89uCjEKY/edit?usp=sharing)[How to install PyTorch on the Duckiebot ](http://docs.duckietown.com/DT18/software_reference/out/pytorch_install.html)
[How to install Caffe and Tensorflow on the Duckiebot](http://docs.duckietown.com/DT18/software_reference/out/caffe_tensorflow_install.html)
[How to use the Neural Compute Stick ](http://docs.duckietown.com/DT18/software_reference/out/ncsdk_how_to.html)Models vs. Data[\[keynote\]](https://github.com/duckietown/lectures/blob/master/2_given/2017-11-29%20-%20ETHZ%20-%20Model%20vs%20Data%20(Julian%20Zilly).key)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2_given/2017-11-29%20-%20ETHZ%20-%20Model%20vs%20Data%20(Julian%20Zilly).pdf) End-to-End Imitation Learning[\[pdf\]](https://github.com/duckietown/lectures/blob/master/2018/Imitation%20Learning.pdf)[Lane control with supervised learning](http://docs.duckietown.com/DT18/opmanual_duckiebot/out/demo_imitation_learning.html)Sim to Real Transfer[\[Google slides\]](https://docs.google.com/presentation/d/1EUgacqcjGKk0jQLJYq-ITmG_x0FFhZUTHZun_FGoH14) ### Human-Machine Interaction and Safety
Introduction to Safety[\[pptx\]](https://github.com/duckietown/lectures/blob/master/3_legacy/C16_intro_to_safety/TransportationRes.pptx)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/3_legacy/C16_intro_to_safety/TransportationRes.pdf) Advanced Safety and Formal Methods[\[pptx\]](https://github.com/duckietown/lectures/blob/master/3_legacy/C17_advanced_safety/Apr6_FormalMethods_standard_v2.pptx)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/3_legacy/C17_advanced_safety/Apr6_FormalMethods_standard_v2.pdf) ### Advanced Perception
Estimation from Motion Blur[\[pptx\]](https://github.com/duckietown/lectures/blob/master/3_legacy/C21_estimation_from_blur/Duckietown%20-%20State%20Estimation%20from%20Blur_ppsplitted.pptx)
[\[pdf\]](https://github.com/duckietown/lectures/blob/master/3_legacy/C21_estimation_from_blur/Duckietown%20-%20State%20Estimation%20from%20Blur_ppsplitted.pdf)
---
### [Access restrictions: regions not served by Duckietown now](https://duckietown.com/legal-policies/access-restrictions/)
**Published:** April 8, 2026
**Author:** Duckietown Admin
**Content:**
# Regional Access Restrictions
United States export control regulations prohibit U.S. businesses, such as Duckietown, from offering services to users in specific sanctioned geographical areas.
To comply with these regulations, Duckietown can not allow users in these countries or regions to access all of, or certain parts of, our website, including aspects of our documentation, pedagogical resources, and code.
These countries or regions are currently: Iran, Sudan, Crimea, Cuba, Syria, North Korea, the Russian Federation, and certain regions of Ukraine. This list is subject to change depending on U. S. export control regulations.
More information about the sanctions programs administered by the Office of Foreign Assets Control (OFAC) of the US Department of the Treasury is available at https://www.treasury.gov/resource-center/sanctions/pages/default.aspx.
Licenses from government authorities would allow us to provide access to educators and learners in some of these regions. However, given the nuanced complexity of the sanctions and their dynamic nature, Duckietown does not have the resources to apply for special licenses. Therefore we must restrict access to our content as it comprises advanced courses in science, technology, engineering, and mathematics.
Depending on your exact location, you may encounter an IP block when attempting to visit or otherwise access our content.
We created Duckietown to democratize access to top-quality (robot autonomy) education and make it accessible worldwide. We do not believe individuals to be generally responsible for the actions of their governments and deeply regret having to use these discriminatory access restriction measures. Nonetheless, *dura lex, sed lex* \[harsh law, but still a law\].
Duckietown will continue to monitor the situation with the U.S. Department of State and Office of Foreign Assets Control closely, and we hope to soon be able to provide access to users across the world while being in compliance with applicable law.
For any additional information, comment, or doubt, please do not hesitate to [reach out](https://www.duckietown.com/contact).
[ Contact us ](/contact)
---
### [Software License](https://duckietown.com/legal-policies/software-license/)
**Published:** April 8, 2026
**Author:** Jacopo Tani
**Content:**
# Duckietown Software Licensing Terms
---
### [Privacy Policy](https://duckietown.com/legal-policies/privacy-policy/)
**Published:** April 7, 2026
**Author:** Duckietown Admin
**Content:**
# Duckietown Services Privacy Policy
---
### [Website accessibility statement](https://duckietown.com/legal-policies/website-accessibility-statement/)
**Published:** February 20, 2026
**Author:** Jacopo Tani
**Content:**
# Website accessibility statement
## Our Commitment to Accessibility
At Duckietown, we believe access to our websites and digital offerings should be available to everyone. As a technology and education company working to shape the next generations of talent worldwide, we aim to make a positive impact in everything we do and to meet the highest standards of social and environmental performance, transparency, and accountability.
We are dedicated to maintaining an accessible online experience for you, and we employ tools to identify areas that need improvement. We strive to support the WCAG 2.2 Level AA, which is recognized as the international standard for web accessibility. While we test our sites to identify and address areas of improvement and work towards a fully accessible experience, some content may not have yet been fully adapted to the accessibility standards.
## Accessibility Assistance
We’re always learning and looking for ways to improve our websites’ accessibility and are committed to making our websites accessible for all instructors and learners.
If you have questions or concerns about accessibility, let us know at [info@duckietown.com](mailto:info@duckietown.com "Email info@duckietown.com").
## VPAT Downloads
---
### [Legal Policies](https://duckietown.com/legal-policies/)
**Published:** April 8, 2026
**Author:** Duckietown Admin
**Content:**
This page provides links to legal policies governing the use of Duckietown products, software, services, and online platforms.
## Core Platform Policies (duckietown.com)
- [Terms and Conditions](https://duckietown.com/terms-and-conditions/ "Duckietown Website Terms and Conditions")
- [Software License](https://duckietown.com/sw-license/ "Duckietown Software License")
- [Privacy Policy](https://duckietown.com/privacy/ "Duckietown Privacy Policy")
- [Access Restrictions](https://duckietown.com/access-restrictions/ "Duckietown Access Restrictions")
- [Accessibility Statement](https://duckietown.com/website-accessibility-statement/ "Duckietown Accessibility Statement")
## Store Policies (get.duckietown.com)
- [Duckietown hardware shipping policy >](https://get.duckietown.com/policies/shipping-policy "Duckietown hardware shipping policy >")
- [Duckietown cancellation policy for subscriptions, pre-orders, trial periods >](https://get.duckietown.com/policies/subscription-policy "Duckietown subscriptions cancellation policy >")
- [Hardware warranty, returns, and refund policy >](https://get.duckietown.com/policies/refund-policy "Duckietown hardware warranty, returns, and refund policy")
- [Duckietown store: privacy policy >](https://get.duckietown.com/policies/privacy-policy "Duckietown store privacy policy >")
- [Duckietown store: Terms of Service >](https://get.duckietown.com/policies/terms-of-service "Duckietown store: Terms of Service >")
---
### [Guides for getting started with Duckietown](https://duckietown.com/guides/)
**Published:** October 15, 2020
**Author:** Duckietown Admin
**Content:**
# Guides for getting started
###### Duckietown is at the intersection of education, training, and research and is designed for instructors, researchers, and makademics. Here are the guides for getting started.
[ START TEACHING ](/guides/start-teaching)
[ START LEARNING ](/guides/start-learning)
[ RESEARCH ](/guides/start-researching)
[ Get hardware ](https://get.duckietown.com/)
[ I want to teach ](/guides/start-teaching)
[ I want to learn ](/guides/start-learning)
[ I want to research ](/guides/start-researching)
[ I want to build ](/get-started)
---
### [People of Duckietown: stories from learners worldwide](https://duckietown.com/news/people-of-duckietown/)
**Published:** March 8, 2022
**Author:** Duckietown Admin
**Content:**
# People of Duckietown
###### We love robots, but humans even more! The experience of the people working with, on, and around Duckietown is what matters most. Here, we get to know the people of Duckietown and their stories.
[ Tell us your story! ](#reach-out-pod)
## The Duckietown stories
[  ](https://www.uniroma1.it/it/pagina-strutturale/home)
[  ](https://www.umontreal.ca/en/)
[  ](https://constructor.university/)
[  ](https://ethz.ch/en.html)
[  ](https://www.uml.edu/)
[  ](https://etu.ru/en/university/)
[  ](https://www.duoc.cl/)
[  ](https://www.bfh.ch/de/)
[  ](https://www.umontpellier.fr/)
[  ](https://www.ualberta.ca/en/index.html)
[  ](https://constructor.university/)
[  ](https://www.grenoble-inp.fr/)
[  ](https://www.uml.edu/)
[  ](https://www.ox.ac.uk/)
[  ](https://web.uniroma2.it/en)
[  ](https://www.bme.hu/en)
[  ](https://www.uao.edu.co/)
[  ](https://www.utoronto.ca/)
[  ](https://www.bme.hu/en)
[  ](https://www.bu.edu/)
[  ](https://www.unr.edu/)
[  ](https://iitj.ac.in/)
[  ](https://www.startpage.com/sp/search)
[  ](https://www.tum.de/)
[  ](https://charleston.edu/)
[  ](https://www.agh.edu.pl/en/)
[  ](https://tu-dresden.de/?set_language=en)
[  ](https://www.massrobotics.org/)
[  ](https://www.uct.ac.za/)
[  ](https://www.hacettepe.edu.tr/english)
[](https://duckietown.com/teaching-robot-autonomy-the-hague-university/)### [ Teaching robot autonomy at The Hague University ](https://duckietown.com/teaching-robot-autonomy-the-hague-university/)
[](https://duckietown.com/learning-robot-autonomy-with-duckiedrones/)### [ Learning robot autonomy with Duckiedrones ](https://duckietown.com/learning-robot-autonomy-with-duckiedrones/)
[](https://duckietown.com/nicolas-figueroa-robotics-in-peru/)### [ Making robotics in Peru more accessible ](https://duckietown.com/nicolas-figueroa-robotics-in-peru/)
[](https://duckietown.com/ozgur-erkent-exploring-robotics-and-rescue-operations-with-duckietown/)### [ Ozgur Erkent: robotic rescue operations with Duckietown ](https://duckietown.com/ozgur-erkent-exploring-robotics-and-rescue-operations-with-duckietown/)
[](https://duckietown.com/intelligent-and-autonomous-mobility-systems/)### [ Intelligent and autonomous mobility systems ](https://duckietown.com/intelligent-and-autonomous-mobility-systems/)
[](https://duckietown.com/future-engineers-with-kevin-smith/)### [ Nurturing future engineers and leaders with Kevin Smith ](https://duckietown.com/future-engineers-with-kevin-smith/)
[](https://duckietown.com/dlugosz-robot-autonomy/)### [ Teaching robot autonomy at AGH Krakow with Prof. Długosz ](https://duckietown.com/dlugosz-robot-autonomy/)
[](https://duckietown.com/dino-claro-duckietown-journey/)### [ Dino Claro: a Duckietown journey from project to thesis ](https://duckietown.com/dino-claro-duckietown-journey/)
[](https://duckietown.com/prof-bruegge-interview-failing-successfully/)### [ Successful failures: learning robotics with Prof. Bruegge ](https://duckietown.com/prof-bruegge-interview-failing-successfully/)
« Previous[Next »](https://duckietown.com/news/people-of-duckietown/2/)
## Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell?
Reach out to us!
First Name
Last Name
Email
I am a
Talk about
Reach out
---
### [Mission: a new vision for education](https://duckietown.com/mission/)
**Published:** May 11, 2021
**Author:** Duckietown Admin
**Content:**
# Mission
###### Duckietown is a new vision for education, starting from robotics and AI
## Mission
Our mission is to make the world excited about the **beauty**, the **fun**, the **importance**, and the **challenges** of **robotics and AI,** through learning experiences that are **tangible**, **accessible**, and **inclusive**.
We achieve this mission by designing robotics platforms and curricula for all levels of education and promoting their use in the world.
### Robotics and AI
##### Beauty
##### Fun
##### Importance
##### Challenges
#### The beauty and the fun
AI and robotics are the most beautiful disciplines – it is mankind’s attempt at creating artificial creatures that think and act like us.
And it is fun to see robots go!
#### The importance and the challenges
AI and robotics will change our world. Everybody should understand the possibilities, the current status and how much is left to do.
### Learning experiences
Although we design, develop and distribute hardware, software and pedagogical materials for the Duckietown platform, this is just a means to an end: we care about the learning experience the platform enables.
#### Tangible
We learn by handling and doing. We believe that to gain real-world competence, especially in robotics, it is necessary to be able to touch a robot.
We join the maker spirit with the academic spirit: the experiences are fun, but there is also a path of academic learning.
#### Accessible
We engineer Duckietown to make the barrier of entry as small as possible: affordable, broadly available, with step-by-step instructions to go from zero to hero.
#### Inclusive
We promote a broad understanding of the effects of robotics and AI in society.
While we do not advocate for everyone to become a roboticist, we believe literacy in the topics of autonomy is needed for everyone, to develop the critical thinking skills for navigating this historical phase of integration of robots in society.
Engineers are building autonomous robots, but society at large will have to decide how much and for what to use them, as the social repercussions of these decisions will be relevant for generations to come.
## The Duckietown leadership
[](https://www.linkedin.com/in/jacopo-tani/)##### [Jacopo Tani, Ph. D.](https://www.linkedin.com/in/jacopo-tani/)
[](https://www.linkedin.com/in/liam-paull-83a5442b/)##### [Prof. Liam Paull, Ph. D. ](https://www.linkedin.com/in/liam-paull-83a5442b/)
### Foundation directors
[](https://www.linkedin.com/in/censi/)##### [Andrea Censi, Ph. D.](https://www.linkedin.com/in/censi/)
[](https://vivo.brown.edu/display/stellex)##### [Prof. Stefanie Tellex (Brown)](https://vivo.brown.edu/display/stellex)
Coordinates the DuckieSky initiative
[](https://www.linkedin.com/in/andreafdaniele/)##### [Andrea Francesco Daniele - Chief Technology Officer](https://www.linkedin.com/in/andreafdaniele/)
[](https://www.ttic.edu/faculty/walter/)##### [Prof. Matthew Walter (TTIC)](https://www.ttic.edu/faculty/walter/)
Coordinates the Duckietown Junior effort
### Address for correspondence
The Duckietown Foundation is registered as a 501(c)(3) non-profit foundation in the state of Massachusetts, USA.
Duckietown Foundation US, Inc.
6 Liberty Square
PMB #217
Boston, MA 02109
---
### [Build your Duckietown robots now with step-by-step instructions](https://duckietown.com/get-started/)
**Published:** November 20, 2018
**Author:** Duckietown Admin
**Content:**
# Duckietown first steps
###### Get started by building your robots and setting up your working environment
## First steps: robot assembly instructions
Already have a Duckietown robot (Duckiebot, Duckiedrone, Duckietown, Autolab)?
Follow the links below to get started building!
Otherwise, [get a Duckietown robot](https://get.duckietown.com/ "Duckietown project online shop").
[ Build my Duckiebot (DB21-J4) ](https://docs.duckietown.com/ente/duckietown-manual/09-db-opmanual-intro/setup-duckiebot-intro.html)
[ Build my Duckietown ](https://docs.duckietown.com/ente/duckietown-manual/30-duckietown-city/duckietowns-intro.html)
[ Build my Duckiedrone (DD24) ](https://docs.duckietown.com/daffy/opmanual-dd24/hw_assembly/dd24-B.html)
Only the latest Duckietown robot models are linked above. If you are looking for assembly and/or operating instructions for older models, [contact us](/contact "Contact Duckietown").
## Software environment setup
Depending on if you are gearing up for learning experiences with Duckiebots or Duckiedrones, choose your path below.
### Duckiebots (DB21-J4)
1\. [Safety first](https://docs.duckietown.com/ente/duckietown-manual/10-setup/03-duckiebot/safety-lithium-ion-duckiebattery.html "Duckiibot and Duckiebattery safety instructions")
2\. [Set up your computer](https://docs.duckietown.com/ente/duckietown-manual/10-setup/setup-introduction.html "Duckietown computer setup instructions")
3\. [Validate your build](https://docs.duckietown.com/ente/duckietown-manual/21-testing-debugging/02-components-testing/index.html "Duckiebot testing and debugging")
4\. [Start learning](https://docs.duckietown.com/ente/duckietown-manual/60-learning-experiences/db-lx-intro.html "Duckietown Learning Experiences") (or enroll in the [MOOC](https://www.duckietown.com/mooc) for a guided experience)
### Duckiedrones (DD24)
1\. [Safety first (drone building)](https://docs.duckietown.com/daffy/opmanual-dd24/preliminaries/safety.html "Duckietown Duckiedrone (DD24) Safety Instructions")
2\. [Safety second too (drone flying)](https://docs.duckietown.com/daffy/course-intro-to-drones/safety/index.html#)
3\. [Software initialization](https://docs.duckietown.com/daffy/opmanual-dd24/preliminaries/software-initialization.html "Duckiedrone software initialization")
4\. [Start learning](https://docs.duckietown.com/daffy/opmanual-dd24/learning-experiences/index.html "DD24 Duckietown Duckiedrone Learning Experiences")
## Where do I start?
Not sure what is the best solution for you?
Duckietown can be used in different ways depending on if you want to teach, learn, or do research.
[ Find my path ](/guides)
---
### [Ducks and robots: why the Duckies in Duckietown](https://duckietown.com/ducks-and-robots/)
**Published:** August 5, 2018
**Author:** Andrea Censi
**Content:**
# Ducks and robots
###### The relationship between rubber ducks and robots might not be obvious. Why the duckies in Duckietown?
## The duckies in Duckietown
We get often asked: Why the duckies?
The are several answers to this question! In no particular order:
##### 1. Duckies against academic plagiarism in robotics research
A well-known challenge in the world of academia is plagiarism, i.e., to take advantage of other people’s work without giving due credit. In simple terms, to copy.
A less well-known but yet important challenge is self-plagiarism, where authors “recycle” (parts of) their work when writing papers, with the objective to maximize the number of publications (“publish or perish!”, they say).
In 2016, to mitigate this phenomenon, [ICRA (the International Conference on Robotics and Automation](https://www.ieee-ras.org/conferences-workshops/fully-sponsored/icra "ICRA: International Conference on Robotics and Automation")), suggested that video submission accompanying articles should include an element of novelty (yes, the duckies!) to make sure those videos were not recycled. It was a hit (see [ICRA 2016 “ducks and Robots”](http://trailer.icra2016.org/ "ICRA 2016 "ducks and Robots"") initiative)!
Here are some examples of what came of it (all credits to the respective authors):
##### 2. Duckies to break common preconceptions on robotics
The most common adjectives that come up when thinking of robots are: powerful, precise, fast, and dangerous. These are attributes that appeal more to specific demographics than others (e.g., males rather than females).
Duckietown’s mission is to democratize access to the science and technology of autonomy. To do so we challenge this preconception, that alienates parts of the population.
We designed Duckietown to instead be the opposite: colorful, imprecise, slow, not powerful, and borderline silly (in appearance), while preserving the technical and scientific complexity of “real*“* robotics. And duckies are inclusive!
##### 3. Duckies as a tribute to the Boston city area
Duckietown was born at the Massachusetts Institute of Technology (MIT), in Cambridge. Ducks play somewhat of a role as a fun symbol of the Boston city area, thanks to the famous children’s story “[Make Way for Ducklings](https://en.wikipedia.org/wiki/Make_Way_for_Ducklings "Make Way for Ducklings")“.
Ducks are immortalized in a statue in the Boston Commons, which has become a city landmark.
---
### [Start doing research with Duckietown: first steps for pros](https://duckietown.com/guides/start-researching/)
**Published:** December 31, 2022
**Author:** Duckietown Admin
**Content:**
# Guide for researchers
###### Build a mobile robotics lab for accessible and reproducible research
## I am a researcher
Duckietown is a modular, inexpensive research platform for studying autonomy in complex systems.
Think of it as an “experimental simulator” which exposes the nuisances of the real world while preserving control over the environment.
### Value proposition
We provide a baseline implementation for you to rapidly and easily test your algorithms on real physical hardware.
- **Convenience:** You can only change the part that interests you and use the rest of the baselines to have a fully functional system.
- **Reproducibility:** Your research has a high impact since it uses a standard platform that others can easily replicate.
### Example tools for research
- Imitation learning template
- Reinforcement learning template
- Database of Duckiebot driving logs
- Duckiebot driving simulator
- Modularized code (ROS baseline template)
- Low cost, standardized robots, and smart city environment
- An international embodied AI competition infrastructure (AI-DO)
- A community to bounce ideas off
- A simulation-based system for robotic agent benchmarking
- A physical system for reproducible agent benchmarking (Autolab)
[
](https://ieeexplore.ieee.org/abstract/document/8372718/)
[
](https://ieeexplore.ieee.org/document/8206612/)
[
](https://arxiv.org/abs/1805.03241)
[
](https://search.proquest.com/docview/1928909929?pq-origsite=gscholar)
[
](https://dspace.mit.edu/handle/1721.1/113135)
[
](https://arxiv.org/abs/1707.07399)
[
](https://www.cs.umb.edu/~craigyu/papers/guidedog.pdf)
[
](https://link.springer.com/chapter/10.1007/978-3-319-55553-9_8)
[
](https://ieeexplore.ieee.org/abstract/document/8190555/)
[ Examples of papers using Duckietown ](/research/papers)
### Researcher: first steps
1. Join the Duckietown community on Slack
2. Learn about the capabilities of the platform
3. Get the hardware
[ 1. Join our Slack ](https://duckietown.com/join-slack)
[ 2. Platform Overview ](/platform)
[ 3. Get the hardware ](https://cutt.ly/website-guides-research-hw)
### Reach out for additional information
Want to upgrade your Duckietown to Autolab and enable experimental evaluations locally, or have questions on how to bring Duckietown to your institution?
Reach out or request a formal quote here:
[ Request a quote ](/request-quote)
[ 3. Check the baselines ](https://cutt.ly/baseline-algorithms)
---
### [Technology: a complete platform for learning robot autonomy](https://duckietown.com/platform/)
**Published:** October 17, 2020
**Author:** Duckietown Admin
**Content:**
# Duckietown Technology
[  ](https://get.duckietown.com/)
###### A modular robotics and AI ecosystem with integrated components designed to provide joyful learning experiences
##### [Hardware](#hardware)
##### [Software](#software)
##### [Simulation](#simulation)
##### [Datasets](#datasets)
##### [Materials](#learning-materials)
Duckietown is a technological platform designed to create, consume, and disseminate robotics and AI learning experiences.
Duckietown integrates the following components:
- [hardware](#hardware)
- [software](#software)
- [simulation](#simulation)
- [learning materials](#learning-materials)
Additional resources:
- [datasets](#datasets)
- evaluation infrastructure
### Hardware: the Duckietown robots
Hardware is the most tangible part of Duckietown: a robotic ecosystem where fleets of autonomous vehicles (Duckiebots) interact with each other and with the urban environment they operate within (Duckietown).
Duckietown was designed to answer the question: what is the **least hardware** we need for deploying single- and multi-robot **advanced autonomy** solutions? In other words, what is the simplest robotic platform that allows us to write scientific papers?
The Duckiebot is a **minimal autonomy platform**. It allows for investigating **complex autonomous behaviors**, **learning** about real-world challenges, and doing **research**.
The main onboard sensor for both Duckiebots and Duckiedrones is a front-facing **camera**, as vision is a tough nut to crack, especially on computationally limited resources.
Duckiebot’s actuators include **two DC motors** (for moving in differential drive configuration) and **RGB addressable LEDs** for signaling to other Duckiebots and shedding light on the road. Additional sensors such as **IMU**, **time-of-flight** and **wheel encoders** are available.
Duckietown is fully decentralized (there are no mega computers doing all the calculations, or eyes in the sky providing guidance), and Duckiebots are fully autonomous. All decision making is done onboard, thanks to the power of **Raspberry Pi** and **NVIDIA Jetson Nano** boards – proper credit card sized computers.
The custom-designed onboard Duckiebattery battery offers **hours of autonomy**, **advanced diagnostics**, and pass-through charging, enabling the Duckiebots to know when it is time to go refill.
[ Get the hardware ](https://get.duckietown.com/)
**Duckietowns** are structured and modular environments built on two layers: road and signal, to offer a repeatable but flexible driving experience, without fixed maps.
The **road layer** is defined by five segment types: straight, curve, 3-, 4-way intersection and empty tiles. Every segment is build on interconnectable tiles which can be rearranged to produce any number of city topographies while maintaining rigorous appearance specifications that guarantee the functionality of the robots.
The **signal layer** in Duckietown is made of traffic signs and road infrastructure.
The signs show both easily machine-readable markers (April Tags) and actual human-readable representations, to provide scalable complexity in perception. Signs enable Duckiebots to localize on the map, interpret the type and orientation of intersections, amongst other uses.
The road infrastructure is made of traffic lights and watchtowers, which are proper robots themselves. At hardware level, traffic lights are Duckiebots without wheels, and watchtowers are traffic lights without LEDs.
Although immobile, the road infrastructure enables every Duckietown to become itself a robot: it can sense, think and interact with the environment.
A Duckietown instrumented with a sufficient number of watchtowers can be transformed in a **Duckietown Autolab**: an accessible, reproducible setup for performance benchmarking of behaviors of fleets of self-driving cars. An Autolab can be programmed to localize robots and communicate with them in real time.
[ Learn about Autolabs ](https://duckietown.com/integrated-benchmarking-and-design-for-reproducible-and-accessible-evaluation-of-robotic-agents/)
#### Duckiebot
- **Sensing**: Camera, Encoders, IMU, Time of Flight, Smart Battery
- **Computation**: Raspberry Pi, Jetson Nano
- **Actuation**: DC motors, LEDs, OLED Screen
- **Memory**: 64GB, class 10
- **Power**: 5V, 10Ah
- **Open design**: supports additional i2c or USB peripherals, e.g., Lidar
- **Open software**: fully transparent software implementation
- [ Get a Duckiebot ](https://get.duckietown.com/products/duckiebot-db21)
[  ](https://get.duckietown.com/products/duckiebot-db21)
#### Duckiedrone
- **Sensing**: Time of Flight, Camera, IMU
- **Computation**: Raspberry Pi 4 - 4GB
- **Actuation**: DC motors, LEDs
- **Memory**: 64GB, class 10
- **Open hardware**: supports additional i2c or USB peripherals
- **Open software**: fully transparent software implementation
- [ Get a Duckiedrone ](https://get.duckietown.com/products/autonomous-raspberrypi-quadcopter-duckiedrone-dd24)
[  ](https://get.duckietown.com/)
#### Duckietown
- **Modularity**: assemble and combine fundamental building blocks
- **Structure**: appearance specifications (colors, geometries) guarantee functionality
- **Smart City**: road and signal layers can be augmented with a network of smart traffic lights and watchtowers to create a real city-robot.
### Software
Duckietown is programmed to be scalable in terms of difficulty level, so that it can adapt to the user’s skill level: from zero to scientist level.
The software architecture in Duckietown allows users to develop on real hardware and simulation and evaluate their agents locally or in remote, on the cloud and on real hardware.
The Duckiebot and Duckietowns can be used via terminal (for experts) or a web duckie-dashboard GUI (for convenience).
The various robotics ecosystem functionalities offered in the base implementation are encapsulated in Docker containers, which include ROS (Robotic Operating System) and Python code.
Docker ensures code reproducibility: whatever “works” at some point in time will continues to work forever as all dependencies are included in the specific application container. In addition, Docker eases modularity, allowing to substitute individual functional blocks without encountering compatibility problems.
Communication between functionalities (perception, planning, high- and low-level control, etc.) is achieved through ROS, a well-known open source middleware for robotics.
Duckietown uses mostly Python, although ROS supports different languages too.
ROS nodes running inside Docker containers rely on a homemade operating system (duckie-OS) that provides an additional layer of abstraction. This step allows running complex sequences with single lines of code. For example, with only three lines of code you can train an agent in simulation, “package” it in a Docker container and deploy it on an actual Duckiebot (real or virtual).
#### Algorithms
- **One code base**: for all users worldwide
- **Many behaviors**: test different perception, control, planning, coordination and ML solutions
- **Collaborative development**: open source, hosted on Github. Anyone can participate in the development.
- **Free autonomy stack**: start learning autonomy in.. full autonomy!
#### Architecture
- **Linux**: start with Ubuntu
- **Docker**: sort of like a set of virtual machines, but more efficient
- **Portainer**: an intuitive way to manage programs (containers) in Docker
- **Duckietown OS**: an extra layer of abstraction to simplify life
- **ROS**: The famous Robotic Operating System manages real-time communications between various components of the software architecture
- **Python** : although other languages are supported, all our code is written in Python
[ Check the developer's book ](https://docs.duckietown.com/daffy/devmanual-software/intro.html)
[ Check the code base ](https://github.com/duckietown)
#### Gym-Duckietown
- **Collaborative development**: the SW in Duckietown is open source, hosted on Github. Anyone can participate in the development.
- **Free**: enough said
- **Physically realistic**
- **Portability**: run the same agents on the physical Duckiebots
- **ML**: built for training and testing AI algorithms
### The Duckietown Simulator
Duckietown includes a simulator, **Gym-Duckietown**, written entirely in Python/OpenGL (Pyglet). The simulator is designed to be physically realistic and easily compatible with the real world, meaning that algorithms developed in the simulation can be ported to physical Duckiebots with a simple click.
In the simulator you can place agents (Duckiebots) in simulated cities: closed circuits of streets, intersections, curves, obstacles, pedestrians (ducks) and other Duckiebots. It can get pretty chaotic!
Gym-Duckietown is fast, open-source, and incredibly customizable. Although the simulator was initially designed to test algorithms for keeping vehicles in the driving lane, it has now become a fully functional urban driving simulator suitable for training and testing machine learning, reinforcement learning, imitation learning, and, of course, more traditional robotics algorithms.
With Gym-Duckietown you can explore a range of behaviors: from simple lane following to complete urban navigation with dynamic obstacles. In addition, the simulator comes with a number of features and tools that allow you to bring algorithms developed in simulation quickly to the physical robot, including advanced domain randomization capabilities, accurate physics at the level of dynamics and perception (and, most importantly: ducks waddling around).
### Experimental datasets
As a complement to the simulation environment and standardized hardware, we provide a database of Duckiebot camera footage with associated technical data (camera calibrations, motor commands) that is continuously updated.
This amount of data is useful both for training certain types of machine learning algorithms (learning by imitation) and for developing perception algorithms that work in the real world.
Given the standardization of the hardware and its international distribution, this database contains information about a multitude of different environmental conditions (light, road surfaces, backgrounds, …) and experimental imperfections, which is what makes robotics fun!
#### The logs
- **Camera calibration**: extrinsic and intrinsic for each Duckiebot
- **Video**: 640x480
- **Motor commands**: time-stamped!
- **Richness of data** : tens of hours, from hundreds of Duckiebots, growing steadily
#### (hidden) The logs
- **Executive**: straightforward instructions for hands-on learning experiences
- **Aligned**: mirroring the provided theory, activities and exercises
- : time-stamped!
- **Richness of data** : tens of hours, from hundreds of robot, growing steadily
### (hidden) Operation Manuals
To accompany Duckietown users in this adventure, we provide detailed step-by-step instructions for all steps of the learning process: from assembling a box of parts to controlling a fleet of self-driving cars roaming around in a smart city.
The spirit of the Duckietown operation manuals, which can be found in the Duckumentation, is to provide executive hands-on directives to get specific things to work. We include graphics, preliminary competence and expected learning outcomes for each section, as well as troubleshooting sections and demo videos when applicable.
### Learning materials
The original subtitle of the Duckietown project was:
*“From a box of components to a fleet of self-driving cars in just 1325 steps, without hiding anything.”*
Since then, the steps might have become more but the spirit has remained the same. All the information to explore, use and develop using Duckietown are in our online library (the “Duckuments”, or “Duckiedocs”).
The documentation provides instructions, operating manuals, theoretical preliminaries, links to descriptions of the codes used, and “demos” of fundamental behaviors, at the level of a single robot or fleet.
The documentation is open source, collaborative (anyone can integrate contributions, which are moderated to ensure the quality), and free.
[ Explore the Duckuments ](https://docs.duckietown.com/)
#### Duckumentation
- **Comprehensive**: from linear algebra to the state of the art in ML
- **Useful**: from theory, to algorithms, to deployment
- **Collaborative**: anyone can contribute!
[ Get started ](/guides)
[ Contact us ](/contact)
---
### [Self driving cars, a technology that could change the world](https://duckietown.com/self-driving-cars-technology/)
**Published:** December 11, 2023
**Author:** Duckietown Admin
**Content:**
# Self Driving cars, a technology that will
change the world
Welcome to the intersection of innovation and mobility – the realm of self driving cars. On this page, we explore autonomous vehicles, their technological intricacies, societal impacts, and the promise of a transformative future they herald for the world of AI robotics.
##### Quick links
- [ Introduction to AVs ](#chap1)
- [ The Technology Behind AVs ](#chap2)
- [ Challenges and Solutions ](#chap3)
- [ Applications Beyond the Road ](#chap4)
- [ Educational Initiatives and Research ](#chap5)
- [ Future Landscape ](#chap6)
- [ Steering towards tomorrow ](#chap7)
## The world of tomorrow
In the following sections, we explore how recent advancements in robotics and autonomous systems technologies promise to forever change the world we live in.
### Introduction to Self Driving Cars
When it comes to transportation, the emergence of self driving cars (sometimes also referred to simply as “autonomous vehicles”, although this is a much broader category) and autonomous technologies represents a revolutionary paradigm shift that transcends traditional notions of mobility.
As urbanization accelerates and traffic congestion becomes an ever-present challenge, self-driving cars have emerged as a promising solution to alleviate the burdens associated with commuting.
This transformative technology not only addresses the perennial issues of time wasted in traffic and the alarming frequency of accidents but also heralds a new era of mobility characterized by efficiency, safety, and reclaimed time.
[ Self-Driving Cars with Duckietown ](/mooc)
###### ##### Addressing the Challenges
One of the most pressing issues faced by urban dwellers is the seemingly inexorable increase in time spent navigating congested roadways.
The advent of self-driving cars offers a beacon of hope in this traffic-laden landscape.
By combining technological advances in sensing, artificial intelligence, and machine learning algorithms, autonomous vehicles can navigate through complex traffic scenarios, in the presence of various real-world nuisances, with unparalleled precision.
The introduction of self-driving cars promises not only a more time-efficient commute, but also a substantial alleviation of the environmental and psychological stressors associated with prolonged hours spent in traffic.
Duckietown was created by a group of researchers in the [Computer Science and Artificial Intelligence Laboratory](https://www.csail.mit.edu/ "CSAIL") (CSAIL) at the [Massachusetts Institute of Technology](https://www.mit.edu/ "MIT") (MIT), to help develop and test cutting-edge algorithms to make self-driving cars work safely and efficiently. Learn more about [Duckietown’s history](https://duckietown.com/history/ "Duckietown's history") here.
###### ##### Safety First
Accidents on roadways have long been a tragic reality of our transportation systems.
Self-driving cars, equipped with state-of-the-art sensor arrays and way faster than human decision-making capabilities, promise to dramatically reduce traffic accidents.
With the elimination of human error, which is a leading cause of accidents, autonomous vehicles have the potential to make road travel significantly safer, saving countless lives and preventing injuries.
Check out MIT Professor Del Vecchio’s [Duckietown lecture on formal design for safety for AVs](https://hubs.ly/Q02bkb_60 "Safe by Design Transportation Systems: present and future"), among the other [autonomous vehicles educational resources](https://duckietown.com/educational-resources/ "Duckietown Educational Materials") provided.
###### ##### Reclaiming Time
Perhaps one of the most intriguing aspects of autonomous driving is the concept of “time reclamation.”
In 2022, the 233 million drivers in the USA alone spent on average 51 hours stuck in traffic. That’s over 1.3 **million** human years of potential wasted stuck in traffic.
As individuals are liberated from the responsibility of constant vigilance behind the wheel, the time traditionally spent navigating through traffic can be repurposed for more meaningful endeavors.
Whether it be catching up on work, engaging in leisure activities, or simply relaxing, the advent of self-driving cars can transform the daily commute from a chore into an opportunity for personal productivity and well-being.
###### ##### The Evolution of Autonomous Mobility
The journey toward autonomous mobility has been marked by rapid technological advancements and collaborative efforts across industries.
From early experiments with basic driver-assistance features to the development of fully autonomous prototypes, the evolution of self-driving cars showcases the resilience and adaptability of technology in the face of complex challenges.
The ongoing collaboration between automotive manufacturers, technology companies, and regulatory bodies is shaping a future where self-driving cars become an integral part of our daily lives.
###### ##### Key Components of Self-Driving Cars
Autonomous vehicles are enabled by a sophisticated blend of technologies.
Sensor arrays, including LiDAR, radar, and cameras, provide real-time data about the vehicle’s surroundings.
Advanced algorithms process this information, and transform it into actionable information, enabling the vehicle to make split-second decisions.
Additionally, connectivity features enable communication between autonomous vehicles and infrastructure, further enhancing safety and efficiency.
###### ##### Implications for Transportation and Urban Planning
The widespread adoption of self-driving cars has far-reaching implications for transportation and urban planning. Cityscapes may evolve as the need for extensive parking facilities diminishes, and roads are optimized for efficient autonomous traffic flow.
Public transportation systems may integrate seamlessly with autonomous vehicles, creating a holistic and interconnected mobility network.
Additionally, the reduction in traffic accidents can lead to lower healthcare costs and improved overall public health.
### The Technology Behind Self Driving Cars
Self-driving cars rely on a blend of hardware and software technologies to continuously, perceive their environment, accurately create a representation of the world and their position within it, update an operation plan on the fly, and execute it safely.
Software algorithms, blending traditional autonomy approaches (planning, estimation and control) to more modern machine learning (ML) agents, process this data in real-time, enabling autonomous vehicles to make split-second decisions and navigate safely on roads.
As we get more and more used to technologies facilitating our driving experience, how long gis the path to complete car autonomy?
[ Duckietown technology platform ](/platform)
###### ##### Levels of Autonomy
At the heart of the self-driving car revolution is the concept of autonomy levels, ranging from Level 0 (no automation) to Level 5 (full automation).
These levels represent the vehicle’s ability to control itself, with Level 5 indicating complete independence from human intervention.
The progression through these levels unlocks a higher degree of automation, [from basic driver-assistance features to fully autonomous operation](https://duckietown.com/educational-resources/ "Duckietown education materials") under diverse and challenging conditions.
###### ##### Hardware
Hardware is the bedrock of every robot, including self-driving cars.
Advanced sensor arrays are key in Autonomous vehicles, and typically include LiDAR and radar, cameras, ultrasonic sensors, as well as a set of more traditional sensing units such as IMUs. Sensors provide the data based on which decisions need to be made, many times per second.
To process the data and transform it into actionable information, there are powerful onboard computers that are required to run the algorithms creating a comprehensive model of the environment, fast enough to operate safely.
Additionally, actuators, such as motors and servos, translate the vehicle’s decisions into physical actions, steering, accelerating, and braking, responsively and with precision.
In Duckietown we designed minimal autonomy platforms: the simplest setup that allows to appreciate the hardest challenges in this field. You can learn more about the [Duckiebot robot kit: Duckietown’s 5th generation model self-driving car](https://get.duckietown.com/products/duckiebot-db21?variant=41543707099311 "Duckiebot with Jetson Nano (DB21-J4)"), on the [Duckietown project online store](https://get.duckietown.com/ "Duckietown project online store").
###### ##### Software
The software architecture of self-driving cars is a multifaceted system responsible for three primary functions: perception, planning and decision-making.
Perception (or, *estimation*) involves interpreting the vast amount of sensor data to understand the vehicle’s surroundings and recognizing obstacles, pedestrians, and other vehicles. [Start learning about estimation algorithms hands-on now with this Duckietown learning experience](https://github.com/duckietown/duckietown-lx/tree/mooc2022/state-estimation "Duckietown state estimation learning experience (LX)").
Planning involves using the representation of the world provided by the perception function, in addition to a notion of global objective, to continuously update the reference trajectory, or desired path, the car should take. [Learn about how robots make plans in this Duckietown learning experience](https://github.com/duckietown/duckietown-lx/tree/mooc2022/planning "Duckietown planning learning experience (LX)").
Decision-making (or, *control*), on the other hand, involves using this the estimate of the vehicle’s pose (i.e., position and orientation) in the world, as well as the relative pose to various obstacles, along with the reference trajectory provided by the planning function to make informed choices, such as adjusting speed and responding to dynamic road conditions. [Try developing a controller yourself with Duckietown’s modeling and control learning experiences](https://github.com/duckietown/duckietown-lx/tree/mooc2022/modcon "Duckietown modeling and control learning experiences (LX)").
The integration of perception, planning, and control algorithms is crucial for the vehicle to navigate complex environments safely, and efficiently and represents the cornerstone of “traditional” robot autonomy technology.
In Duckietown, we provide hands-on, state-of-the-a-rt, [interactive learning experiences](https://github.com/duckietown/duckietown-lx "Duckietown learning experiences (LX)") for anyone to delve in the details of each of these steps (and more).
###### ##### Integration of AI and Machine Learning
Central to the autonomy of self-driving cars is the integration machine learning agents, often referred to generalistically as “AI” (artificial intelligence).
These “AI” algorithms change the approach with respect to traditional robotics enabling the vehicle to learn from vast datasets, improving their ability to interpret and respond to diverse scenarios.
ML agents require as much “good” training data as possible to be able to tackle unforeseen or “corner cases” scenarios when deployed in the real world. What “good” means is a subject of current research, with an important debate (“sim-to-real transfer”, or “sim2real”) developing around the question of whether, or rather how much, training an ML agent in a virtual simulation can lead to learning that generalize to the real world applications.
In Duckietown, we developed a simulation environment, [Gym-Duckietown](https://github.com/duckietown/gym-duckietown "Gym Duckietown simulator"), equipped with domain randomization and other features to facilitate ML agent training and deployment on real robots.
### Challenges in Self Driving Cars Technology
The pursuit of self-driving technology, while promising changes in transportation, is not without its share of challenges.
From technical hurdles to ethical considerations, and the need for robust regulatory frameworks, the development and deployment of autonomous vehicles are navigating uncharted territory.
A necessary step for facilitating the development of self driving technology is to make quality education materials accessible to all, which is Duckietown’s ongoing mission. The Duckietown materials and technological platform are used:
- to teach, mostly at university level
- to learn, mostly online by industry professionals
- to do research, mostly rapid prototyping in mobile robotics science and technology.
[ Duckietown education materials ](/educational-resources)
###### ##### Technical Hurdles
Self-driving cars are a safety-critical application, hence every technical challenge becomes important.
From perfecting sensors, to refining algorithms so to increase safety, to developing risk models, autonomous vehicles are complex systems where each component influences the behavior and outcomes of the others.
One of biggest challenges is related to the tackling of the so-called (real-world) “nuisances”, i.e., those corner cases that happen rarely. In other words, making the technology *robust.*
Driving scenarios are incredibly diverse and dynamic, demanding systems that can adapt to unpredictable situations. Factors such as inclement weather, road construction, the color and materials of the surrounding vehicles, and the need for precise mapping contribute to the complexity of creating foolproof autonomous systems.
Continuous advancements in hardware, software, and connectivity are essential to overcome these hurdles. Robust testing methodologies, including simulations and extensive real-world trials, are crucial to validate the performance of self-driving systems across diverse conditions.
With the ability to architect and build smart cities with arbitrary topographies, [Duckietown serves as an ideal “experimental simulator”](https://duckietown.com/integrated-benchmarking-and-design-for-reproducible-and-accessible-evaluation-of-robotic-agents/ "Duckietown Autolabs"), sitting in between simulation only (where things are “doomed to succeed”), and real-world deployment, which is often complex: costly and risky.
###### ##### Ethical Considerations and Safety Challenges
Ethical considerations surrounding decision-making algorithms come to the forefront as the technology matures to the point of making broad implementation possible.
The “trolley problem” is a notable example, where the algorithm must make split-second decisions in the event of an unavoidable accident, potentially impacting the occupants or pedestrians.
Resolving these ethical dilemmas is intricate, requiring a delicate balance between prioritizing human safety, adhering to legal standards, and minimizing harm.
Transparency in the decision-making processes and public engagement in shaping ethical guidelines can contribute to building trust and addressing these ethical considerations.
Even for this reason, Duckietown supports the dissemination of self-driving car science and technology, making accessible to all worldwide. While not everyone should become a scientist or an engineer in these fields, we believe every citizen should have sufficient understanding of what goes on in the “mind of a robot” to be able to take informed decisions while contributing to the public debate.
###### ##### Regulatory Frameworks and Standards
The absence of comprehensive regulatory frameworks and standards poses a significant challenge to the widespread adoption of self-driving technology.
Governments and regulatory bodies face the task of creating legislation that ensures the safety of autonomous vehicles without stifling innovation.
Striking the right balance between allowing experimentation and setting stringent safety standards is a delicate process. Establishing clear guidelines for testing, certification, and operation of self-driving cars is imperative to mitigate risks and ensure harmonized integration into existing transportation systems.
International collaboration is also vital to establish consistent global standards that facilitate the interoperability of autonomous vehicles across borders.
###### ##### Cybersecurity Concerns
As self-driving cars become increasingly connected, the vulnerability to cybersecurity threats becomes a critical concern.
The reliance on software systems and communication networks exposes autonomous vehicles to potential hacking, jeopardizing not only the safety of occupants but also the integrity of transportation systems.
Developing robust cybersecurity measures, including encryption, secure communication protocols, and constant monitoring, is essential to safeguard self-driving cars from malicious attacks.
###### ##### Public Perception and Acceptance
The success of self-driving technology is not solely dependent on its technical capabilities but also on public perception and acceptance.
Building trust in the reliability and safety of autonomous vehicles requires effective communication, education, and transparency.
Addressing concerns related to job displacement, privacy, and the perceived loss of control is essential for garnering public support. Collaborative efforts between industry stakeholders, government agencies, and advocacy groups can help shape a positive narrative and facilitate a smoother transition to a self-driving future.
### Applications Beyond the Road
The impact of self-driving technology extends far beyond the confines of roadways, promising to reshape industries, enhance public services, and usher in a new era of smart mobility.
Here, we explore the diverse applications of self-driving cars, ranging from industrial and commercial uses to their role in emergency services and their transformative impact on public transportation.
Robot autonomy technologies are increasingly finding their way in military applications too, shaping the course of ongoing conflicts.
[ Learn about Duckiedrones ](https://docs.duckietown.com/daffy/opmanual-duckiedrone/intro.html)
###### ##### Industrial and Commercial Applications:
Self-driving technology holds immense potential for revolutionizing industrial and commercial sectors. In logistics and transportation, autonomous vehicles can optimize supply chain operations by streamlining the movement of goods. Automated delivery trucks can operate efficiently, reducing costs and increasing the speed of deliveries. In warehouses, autonomous robots can navigate and transport goods, enhancing efficiency and minimizing manual labor. The flexibility and precision of self-driving technology make it a valuable asset in industries where the movement of goods is a critical component.
###### ##### Impact on Public Transportation:
The integration of self-driving cars into public transportation systems promises a paradigm shift in urban mobility. Autonomous buses and shuttles can provide cost-effective and environmentally friendly alternatives to traditional public transport. These vehicles can operate on fixed routes or dynamically adapt to demand, optimizing efficiency and reducing congestion. The last-mile connectivity problem can be addressed through autonomous ride-sharing services, providing seamless connections between public transit hubs and passengers’ final destinations. As a result, public transportation becomes more accessible, responsive, and capable of meeting the evolving needs of urban populations.
###### ##### The Role of Self-Driving Cars in Emergency Services:
Self-driving cars have the potential to revolutionize emergency services by enhancing response times and providing critical support in crisis situations. Autonomous vehicles equipped with medical supplies can function as mobile clinics, reaching remote or disaster-stricken areas where immediate medical attention is crucial. In emergencies, self-driving ambulances can navigate through traffic more efficiently, expediting the transportation of patients to medical facilities. Furthermore, autonomous vehicles can serve as mobile command centers for emergency responders, offering a versatile and rapidly deployable resource in crisis situations.
###### ##### Smart Mobility:
The concept of smart mobility encompasses the seamless integration of various transportation modes through intelligent, data-driven systems. Self-driving cars play a pivotal role in realizing smart mobility by contributing to a connected and efficient transportation network. Through vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication, autonomous vehicles can share real-time data, optimizing traffic flow and reducing congestion. Smart city initiatives can leverage self-driving technology to create responsive and sustainable urban environments, where transportation is not only efficient but also environmentally conscious.
### Educational Initiatives and Research in Autonomous Mobility
The field of autonomous mobility is witnessing a surge in [educational](https://duckietown.com/guides/start-researching/ "Duckietown start researching") initiatives and research endeavors as academia, industry, and government agencies collaborate to propel advancements in self-driving technology. Several prominent research initiatives are pushing the boundaries of knowledge and technology, paving the way for the widespread adoption of autonomous vehicles.
###### ##### Waymo - Google's Self-Driving Car Project:
Waymo, a subsidiary of Alphabet Inc. (Google’s parent company), has been at the forefront of autonomous mobility research. With a focus on developing self-driving technology, Waymo’s research encompasses a range of areas, including sensor technologies, machine learning algorithms, and real-world testing. Waymo’s work has laid a foundation for many aspects of autonomous vehicle development.
###### ##### MIT-AIMM - Autonomous Intelligent Machines and Mobility:
MIT’s Autonomous Intelligent Machines and Mobility (AIMM) initiative is dedicated to advancing the science and technology behind autonomous mobility. Researchers at MIT-AIMM explore topics such as perception, decision-making, and human-machine interaction to create more intelligent and adaptable autonomous systems. The initiative fosters interdisciplinary collaboration to address the multifaceted challenges in autonomous mobility.
###### ##### Stanford Autonomous Systems Lab:
Stanford University’s Autonomous Systems Lab is actively engaged in research related to autonomous vehicles and robotics. The lab focuses on developing perception and control algorithms that enable vehicles to navigate complex environments safely. Their research includes applications in self-driving cars as well as unmanned aerial vehicles (UAVs), showcasing a broad approach to autonomous systems.
###### ##### UC Berkeley DeepDrive:
The DeepDrive initiative at the University of California, Berkeley, delves into the intersection of deep learning and autonomous driving. Researchers at DeepDrive focus on creating advanced neural network architectures for tasks such as object detection and path planning. Their work aims to enhance the capabilities of autonomous vehicles through state-of-the-art machine learning techniques.
###### ##### Apollo by Baidu:
Baidu’s Apollo project is an open-source platform for autonomous driving. The initiative focuses on collaborative research, providing a platform for researchers and developers to contribute to the development of autonomous vehicle technologies. Apollo’s goal is to accelerate the progress of autonomous driving through shared knowledge and resources.
###### ##### NVIDIA Autonomous Vehicles Research:
NVIDIA, a leader in graphics processing units (GPUs), is actively involved in research and development for autonomous vehicles. Their work encompasses hardware solutions for AI computing, including advanced GPUs optimized for deep learning. NVIDIA collaborates with various partners to drive innovation in autonomous mobility.
### The future landscape of self driving cars
As self-driving technology continues to add new pieces to its [history](https://duckietown.com/history/ "Duckietown history"), the future landscape of [self-driving cars](https://duckietown.com/mooc/ "Duckietown MOOC") promises a transformational shift in transportation, with implications reaching far beyond the roads we travel. From ongoing initiatives and future projects to the societal, economic, and environmental implications, the future of self-driving cars holds the potential to redefine how we move, work, and interact with our surroundings.
###### ##### Current Initiatives and Future Projects:
The landscape of self-driving cars is characterized by a multitude of ongoing initiatives and future projects. Major players in the automotive and technology industries, such as Waymo, Tesla, and traditional automakers, are investing heavily in the research and development of autonomous vehicle technologies. Ongoing projects include the deployment of autonomous ride-sharing services, the testing of self-driving trucks for freight transportation, and the exploration of advanced sensor technologies to enhance perception and decision-making capabilities. Future projects are likely to focus on increasing the sophistication of autonomous systems, expanding into new geographical regions, and integrating self-driving technology with emerging smart city initiatives.
###### ##### Societal and Economic Implications:
The widespread adoption of self-driving cars carries significant societal and economic implications. On the societal front, autonomous vehicles have the potential to enhance mobility for individuals with disabilities and the elderly, providing newfound independence. The shift to self-driving technology may also redefine the concept of car ownership, with shared autonomous fleets reducing the need for private vehicle ownership in urban areas. Economically, the self-driving industry is poised to create jobs in research and development, software engineering, and maintenance, while simultaneously disrupting traditional employment sectors, such as driving-based professions. Striking a balance between job creation and displacement will be a critical aspect of managing the societal and economic impact of autonomous mobility.
###### ##### Environmental Implications and Sustainability:
The environmental implications of self-driving cars are multifaceted. On one hand, the potential for more efficient traffic flow and optimized driving patterns could lead to reduced fuel consumption and lower emissions. Moreover, the electrification of autonomous fleets could contribute to a more sustainable transportation ecosystem. However, the increased reliance on technology and the production of autonomous vehicles may pose environmental challenges related to resource extraction, manufacturing processes, and electronic waste. Balancing the environmental benefits and challenges will be crucial to ensuring that the future landscape of self-driving cars aligns with broader sustainability goals.
###### ##### Timeline for Mass Deployment:
The timeline for the mass deployment of self-driving cars remains a topic of speculation, contingent on the resolution of technical, regulatory, and societal challenges. While certain cities and regions are witnessing pilot programs and limited deployments, achieving widespread adoption requires addressing safety concerns, refining technology, and establishing comprehensive regulatory frameworks. Experts suggest that various levels of autonomy, particularly in controlled environments or specific use cases, may become more prevalent in the near term. However, achieving Level 5 autonomy, where vehicles can operate without human intervention under all conditions, may take more time and depend on continued advancements in technology and regulatory acceptance.
### Steering towards tomorrow
As we conclude this exploration, several key insights emerge, pointing towards a [future](https://duckietown.com/mission/ "Duckietown mission") where autonomy in mobility becomes a defining aspect of our daily lives.
### Learn more about Duckietown
The [Duckietown](https://duckietown.com/ "Duckietown website main page") platform enables state-of-the-art [robotics and AI learning experiences](https://github.com/duckietown/duckietown-lx "Duckietown learning experiences (LX)").
It is designed to help [teach](https://duckietown.com/guides/start-teaching/ "Duckietown starter guide for teachers"), [learn](https://duckietown.com/guides/start-learning/ "Duckietown starter guide for learners"), and [do research](https://duckietown.com/guides/start-researching/ "Duckietown starter guide for researchers"): from exploring the [fundamentals of computer science and automation](https://duckietown.com/educational-resources/ "Duckietown education materials") to [pushing the boundaries of human knowledge](https://duckietown.com/research-papers/ "Duckietown research papers").
[ Get started with Duckietown! ](https://www.duckietown.com/guides)
[ Read the Duckietown stories ](https://www.duckietown.com/news/people-of-duckietown)
---
### [Start learning with Duckietown: first steps for makademics](https://duckietown.com/guides/start-learning/)
**Published:** December 31, 2022
**Author:** Duckietown Admin
**Content:**
# Guide for learners
###### Experience the same education as the world’s best engineering colleges, for free, and get support from our international community of learners, professionals and professors
[ Start Learning with Duckietown ](#learner-get-started)
## Learn robotics and AI autonomously
Want to learn autonomy.. in autonomy? You might be a Makademic!
Makademics are a fusion of “makers” and “academics” who want to learn and build on their own, outside of an educational institution, and also want a deep understanding of why and how things are working (yes, we just made this up).
 This 2018 Duckietown student became a Professor at MIT, in Boston, in 2023
 This 2019 Duckietown student is now a senior research engineer at Motional, a self-driving car company, in Boston in 2021
You can learn about robotics and AI by using all of the course materials we have to offer, at your own pace.
We created an online course to get your learning adventure in Duckietown started.
Gain access to a unique hands-on learning experience, and an international community to bounce off ideas with.
We recommend you consider getting a Duckietown, Duckiebot, and/or Duckiedrone for an even more effective learning experience.
All Duckietown hardware is designed to work with the ecosystem resources (code, activities, exercises, simulation, etc.) out of the box, allowing you to focus on learning about modern robotics challenges right away.
 This 2017 Duckietown student became a Professor at ETH Zurich, in Switzerland, in 2020
[ Subscribe to the Duckietown newsletter ](/contact#stay-in-touch)
### Makademic first steps
1. Join the [**Duckietown Slack**](https://duckietown.com/join-slack "Duckietown Slack") and gain immediate access to the Duckietown community.
2. Enroll in the **free** massive open online course (MOOC) “[**Self-Driving Cars with Duckietown**](https://www.duckietown.com/mooc "Self-Driving Cars with Duckietown")” on edX. You will find instructions to start setting up your working environment within.
3. Consider getting a [**Duckiebot MOOC starter kit**](https://cutt.ly/website-hw-learner-guide "Duckiebot MOOC starter kit") to learn using real hardware.
[ 1. Join our Slack ](https://duckietown.com/join-slack)
[ 2. Enroll in the free Duckietown Online Course ](https://www.duckietown.com/mooc)
[ 3. Get the hardware ](https://cutt.ly/website-hw-learner-guide)
If you already have a Duckiebot, Duckiedrone, and/or Duckietown and are looking for instructions to start building and setting up your working environment:
[ Start building ](/get-started)
---
### [Start teaching with Duckietown: first steps for instructors](https://duckietown.com/guides/start-teaching-with-duckietown/)
**Published:** December 31, 2022
**Author:** Duckietown Admin
**Content:**
# Guide for instructors
###### Teach top robot autonomy classes while saving time and minimizing uncertainty
##### [Why Duckietown?](#guide-teach-why)
##### [The Class-in-a-box](#guide-teach-resources)
##### [How do I start?](#guide-teach-start)
## Instructor benefits
Duckietown is a teaching environment for creating and delivering learning experiences in robotics and AI-related topics.
We provide a “one-click” comprehensive set of tools to support your teaching efforts: hardware, software, curricula, evaluations of outcomes, and support.
Duckietown has been developed throughout the years, starting at MIT, with the feedback of hundreds of university professors and thousands of learners of all skill levels.
We offer a pedagogical infrastructure that reduces the complexity and time needed to prepare, run, and grade hands-on robot autonomy classes.
- **Save time**: before, during and after the course
- **Guided sandbox**: start simple then dive deep
- **Peace of mind**: premium support and fulfillment
- **Community**: thousands of dedicated professionals
- **Doctrine of intersections**: use your Duckietown lab for doing research and student projects while not teaching
### Teaching with Duckietown
We want every student to gain a deep understanding and practical knowledge of state-of-the-art problems, the approaches to solving them, and the tools needed to get the job done.
Duckietown’s platform is designed to empower you to deliver these experiences.
[  ](#guide-teach-start)
[ Say no more, where do I start? ](#guide-teach-start)
## Resources
Duckietown offers syllabi, lecture videos, slides, Jupyter notebook activities, exercises, remote grading infrastructure, different robots, a dedicated simulator, a software development environment, open-source autonomy code, a knowledge base, operation manuals, and access to communities for learners and instructors.
You can access most of these resources for free and evaluate them before committing to teaching with Duckietown. In addition to technical components, we offer premium support packages for instructors and teaching assistants to minimize the stress of teaching a robotics class.
Take a moment now to review the components of the technical platform, and do not hesitate to reach out if you have any questions or doubts.
[ Technical platform ](/platform)
[ Reach out for help ](#teacher-reach-out)
### The Class-in-a-box
Setting up a state-of-the-art robot autonomy class is hard and time-consuming.
The Duckietown class-in-a-box is designed to get you started quickly and painlessly, while providing broad avenues for customization as you become familiar with the provided baselines and workflows.
###### Build your curriculum
Choose from a set of existing [modules](/instructors/classes/educational-resources "Duckietown educational resources") and reinforce the curriculum through the Duckietown [platform](/platform "Duckietown technical platform components"), or have your students take our ready-to-go [online course](/mooc "Self-Driving Cars with Duckietown massive open online course (MOOC)") as a flipped classroom.
###### (Optional) Customize the modules to your needs
Modify the slides, learning activities, and exercises to emphasize what you like.
###### Choose your teaching activities
Each module contains:
- interactive activities in simulation or on hardware,
- an exercise without a solution, to be submitted by each student.
###### (Optional) Get the hardware
Learning robot autonomy with real robots is better than only using simulation.
###### (Optional) Get a Duckietown Professor subscription
Minimize the stress of giving a class by signing up for a Professor subscription, and get priority technical support for you and your teaching assistants, amongst other perks.
Community-based support is always available through our Slack and Stack Overflow spaces.
###### Grade the homework exercises
Use our online [evaluation infrastructure](https://challenges.duckietown.org/v4/ "Duckietown Challenges Server") for automated performance scoring.
(Advanced users can even build a [Duckietown Autolab](/platform#hardware "Duckietown platform: hardware") to automate grading through hardware evaluations).
###### (Optional) Engage students in projects
We provide mechanisms for you to guide your students to create more complex projects in teams.
[ Example class resources ](/instructors/classes/educational-resources)
[ Learning Experiences ](https://github.com/duckietown/duckietown-lx)
## One platform for many experiences
The class-in-a-box can be used in several ways:
1. As a **class resource**: to teach robot autonomy courses in a traditional university setting, both as a standalone Duckietown course or as support to existing curricula;
2. As a **flipped classroom**: where students watch the “Self-Driving Cars with Duckietown” massive open online course from home, and you use the class time to work through the hands-on learning activities;
3. As an **experimental platform**: where the hardware and software are used as the laboratory component of another class, or for doing research.
[ Learn about the "Self-Driving Cars with Duckietown" MOOC ](/mooc)
### About the community
When you bring Duckietown to your institution, your students will be joining a global community which includes opportunities for worldwide collaboration and competition.
We offer an instructors-only community too, to share experiences and resources on teaching with Duckietown.
You can read interviews to Professors, students and professionals in our “people of Duckietown” page.
[ Read about the people of Duckietown ](https://www.duckietown.com/news/people-of-duckietown)
[ Join our learner community on Slack ](https://duckietown.com/join-slack)
## Instructor: first steps
Want to start teaching with Duckietown?
1. Check out the instructor manual;
2. Try the student experience in simulation;
3. Reach out for a demo-kit discount code.
[ 1. Instructor Manual ](https://docs.duckietown.com/daffy/instructor-manual/intro.html)
[ 2. Check the LXs ](https://github.com/duckietown/duckietown-lx)
[ 3. Reach Out ](/request-quote)
If you already have a Duckiebot, Duckiedrone, and/or Duckietown and are looking for instructions to start building and setting up your working environment:
[ Start building ](/get-started)
### Reach out for additional information
Not sure what fits your particular use case, or need a formal quote?
For any questions or doubts, do not hesitate to reach out.
[ Request a quote ](/request-quote)
Teaching the Duckietown class was a wonderful experience for me and my students. The materials are great and the hands-on experience with the robot really helps reinforce the curriculum.
![Matthew R. Walter, Prof.]()
Matthew R. Walter, Prof.Robot Intelligence through Perception Laboratory (RIPL)
Toyota Technological Institute at Chicago
Starting from MIT CSAIL, Duckietown has grown into a global initiative that is inspiring students around the world to learn about self-driving cars, as well as the science and engineering of autonomy.
![Daniela Rus, Prof.]()
Daniela Rus, Prof.Director, Computer Science and AI Lab (CSAIL)
Massachusetts Institute of Technology (MIT)
The Duckietown class is the autonomous driving pie: the filling is hardcore robotics, the casing is artificial intelligence, and as a plus, you get some funny ducks on top!
![Manfred Diaz]()
Manfred DiazPh. D. Candidate
University of Montreal
If University were like learning how to play a new instrument, where lessons are the exercises and exams the final auditions, Duckietown would be the full-blown rock concert, where you play for your fans and look to your heroes with admiration.
![Gioele Zardini]()
Gioele ZardiniPh. D. Candidate
ETH Zurich
Duckietown was much more than just a class, it was a hands-on deep dive into hardware, software, and systems integration, and, most of all, it was a blast!
![Teddy Ort, Ph. D.]()
Teddy Ort, Ph. D.Senior Director
Robot Perception and AI at Symbotic
Spending the summer in Duckietown at MIT made me discover a completely new world: I understood that education can be a game and learning can be fun!
![Valeria Cagnina]()
Valeria CagninaYoung Enterpreneur
Great platform and solid e-learning experience!
![Benjamin Sawicki]()
Benjamin SawickiCoordinator Knowledge & Technology Transfer
NCCR Automation
Highly recommended. I joined the first cohort of "Self-Driving Cars with Duckietown" and loved it.
![Manuel Heredia Ortiz]()
Manuel Heredia OrtizVice President, PhD, Executive MBA
Airbus
Very excited to be a part of the upcoming edition of "Self-Driving Cars with Duckietown". In the last edition, my students and I enjoyed learning of many new topics with hands-on experience. All the student team had a very good experience. Amazing support by Duckietown!
![Kishanprasad Gunale]()
Kishanprasad GunaleAssitant Professor
MIT College of Engineering, Pune
Thanks to the extraordinary learning and research platform Duckietown from ETHZurich, TTI Chicago, University of Montreal and the helpful around-the-hour support from the Duckietown core team #BFH was able to successfully introduce its BSc automotive engineering students in the exciting world of autonomous driving!
![Peter Affolter, Prof.]()
Peter Affolter, Prof.Head of Department Automotive Engineering
Berner Fachhochschule BFH
I believe that there’s a lot of interesting research directions that come from a standardized, small scale, accessible autonomous driving platform like Duckietown.
![Liam Paull, Prof.]()
Liam Paull, Prof.Department of Computer Science and Operations Research
Université de Montréal
All the different concepts ranging from control to localization to computer vision can be applied in Duckietown. My students like it and they learn a lot about robotics.
![Francesco Maurelli, Prof.]()
Francesco Maurelli, Prof.Jacobs University of Bremen
In my class we go over the basics of robotics, starting with multi-agent processing or multi-process systems, like most robots are these days, and the Duckiebots are perfect for that
![Paul Robinette, Prof.]()
Paul Robinette, Prof.Department of Electrical and Computer Engineering
University of Massachusetts Lowell
I think Duckietown is a very good education platform for teachers. We make use of the very good materials provided by Duckietown and I’m very satisfied with its implementation.
![Lei Yang, Prof.]()
Lei Yang, Prof.Department of Computer Science and Engineering
University of Nevada, Reno
In engineering, true learning comes from practical implementation, and Duckietown offers that opportunity effectively.
![Shima Akbari]()
Shima AkbariPh. D. Candidate
University of Rome "La Sapienza"
We have received large positive feedback for Duckietown from students and researchers who have used it at IIT Jodhpur, with many appreciating its entertaining and challenging nature.
![Debasis Das, Prof.]()
Debasis Das, Prof.IIT Jodhpur
Duckietown has allowed students from different campuses of the school of computer science and telecommunications at Duoc UC to immerse themselves in the world of robotics and autonomous vehicles, Linux, ROS, and Python, in an accessible and exciting way.
![Félix Donoso H., Prof.]()
Félix Donoso H., Prof.School of Computer Science and Telecommunications at the Duoc UC Professional Institute
I haven't found another hardware platform as good as Duckietown for my needs. It's a simple platform with a fun approach, well-documented, and with a really reactive community. Even compared to other commercial products, Duckieown stands out. It fulfills all the needs from beginners to experts.
![Peter Affolter, Prof.]()
Peter Affolter, Prof.Berner Fachhochschule (BFH)
---
### [Robotics and AI research with Duckietown](https://duckietown.com/research-papers/)
**Published:** July 17, 2018
**Author:** Duckietown Admin
**Content:**
# Research papers
###### Duckietown is a versatile and accessible mobile robotics and embedded AI modular setup for reproducible scientific research.
[ Tell us about your research ](#research-reach-out)
Email
Link to paper
Tell us about your research
[](https://duckietown.com/visual-control-for-autonomous-navigation-in-duckietown/)### [ Visual Control for Autonomous Navigation in Duckietown ](https://duckietown.com/visual-control-for-autonomous-navigation-in-duckietown/)
[](https://duckietown.com/sim2real-lane-segmentation-via-domain-adaptation/)### [ Sim2Real Lane Segmentation via Domain Adaptation ](https://duckietown.com/sim2real-lane-segmentation-via-domain-adaptation/)
[](https://duckietown.com/interpretable-reinforcement-learning-for-visual-policies/)### [ Interpretable Reinforcement Learning for Visual Policies ](https://duckietown.com/interpretable-reinforcement-learning-for-visual-policies/)
[](https://duckietown.com/visual-feedback-for-lane-tracking-in-duckietown/)### [ Visual Feedback for Autonomous Lane Tracking in Duckietown ](https://duckietown.com/visual-feedback-for-lane-tracking-in-duckietown/)
[](https://duckietown.com/reproducible-sim-to-real-traffic-signal-control-environment/)### [ Reproducible Sim-to-Real Traffic Signal Control Environment ](https://duckietown.com/reproducible-sim-to-real-traffic-signal-control-environment/)
[](https://duckietown.com/adapting-world-models-with-latent-state-dynamics-residuals/)### [ Adapting World Models with Latent-State Dynamics Residuals ](https://duckietown.com/adapting-world-models-with-latent-state-dynamics-residuals/)
[](https://duckietown.com/transformer-visual-control-for-dynamic-obstacle-avoidance/)### [ Transformer Visual Control for Dynamic Obstacle Avoidance ](https://duckietown.com/transformer-visual-control-for-dynamic-obstacle-avoidance/)
[](https://duckietown.com/vae-based-out-of-distribution-detectors-for-embedded-systems/)### [ VAE-Based Out-of-Distribution Detectors for Embedded Systems ](https://duckietown.com/vae-based-out-of-distribution-detectors-for-embedded-systems/)
[](https://duckietown.com/semantic-image-segmentation-methods-in-duckietown/)### [ Semantic Image Segmentation Methods in Duckietown ](https://duckietown.com/semantic-image-segmentation-methods-in-duckietown/)
« PreviousPage1[Page2](https://duckietown.com/research-papers/2/?doing_wp_cron=1786646276.4517979621887207031250)[Page3](https://duckietown.com/research-papers/3/?doing_wp_cron=1786646276.4517979621887207031250)[Page4](https://duckietown.com/research-papers/4/?doing_wp_cron=1786646276.4517979621887207031250)[Page5](https://duckietown.com/research-papers/5/?doing_wp_cron=1786646276.4517979621887207031250)[Next »](https://duckietown.com/research-papers/2/)
A more comprehensive list of research papers is available on [Google Scholar](https://scholar.google.com/scholar?hl=it&as_sdt=0%2C5&q=duckietown&oq=duckietown).
## Tell us about your research
If you would like your paper to be featured on this page, let us know about it!
---
### [Contact information: reach out to us](https://duckietown.com/contact/)
**Published:** October 4, 2020
**Author:** Duckietown Admin
**Content:**
# Contact the Duckietown team
## Technical Support
To receive technical support:
1\. [Join the Duckietown Slack channel](https://duckietown.com/join-slack "Join the Duckietown Slack")
2\. If you have been assigned a priority support channel, ask there. If you want a priority support channel and you are using Duckietown for teaching or research, reach out to us.
3\. Slack will greet you with instructions on how to access the [Duckietown Stack Overflow](https://stackoverflowteams.com/c/duckietown/questions)
4\. Search on Stack Overflow for existing accepted answers to your question. If none is available, post a question.
5\. Take the question link and post it in one of the #help- channels on Slack. If you are not sure which one to use, post your link in [\#how-to-get-help](https://duckietown.slack.com/archives/CHHQJ0E0H)
## General Enquiries
For general questions, comments, or feedback, reach out to us at .
To request a quote, fill out [this form](https://contact.duckietown.com/request-a-quote) instead.
For technical support, read above.
[ Send us an email ](mailto:info@duckietown.com?subject=Website%20Enquiry)
[ Request a quote ](https://contact.duckietown.com/request-a-quote)
## Stay in touch
Sign up for our mailing list below to receive quasi-monthly updates on the evolution of Duckietown.
Service announcements (e.g., planned downtime for maintenance) are delivered on Slack, so make sure to sign up!
We shall not spam, nor have you spammed. We will only use your personal information to administer your account and provide the products and services you requested from us. From time to time, we will contact you about our products and services, as well as other Duckietown-related content that may be of interest to you.
### Sign up to the Duckietown newsletter
Subscribe to our newsletter to get the latest updates about Duckietown!
First Name
Last Name
Email
Subscribe
## Correspondance
This is our mailing address:
Duckietown, Inc.
8 The Green, STE A
19901, Dover, DE
USA
The best way to interact with us is through the Duckietown Slack. If you are not a member of the community yet, it is best to [send us an email](#reach-out "General Enquiries for Duckietown").
---
### [Duckietown Terms and Conditions](https://duckietown.com/legal-policies/terms-and-conditions/)
**Published:** August 2, 2018
**Author:** Duckietown Admin
**Content:**
# Terms and Conditions of Use
---
### [Mission: democratizing robot autonomy](https://duckietown.com/mission-content/)
**Published:** May 14, 2018
**Author:** Duckietown Admin
**Content:**
## Mission
Our mission is to make the world excited about the **beauty**, the **fun**, the **importance**, and the **challenges** of **robotics and AI,** through learning experiences that are **tangible**, **accessible**, and **inclusive**.
We achieve this mission by designing robotics platforms and curricula for all levels of education and promoting their use in the world.
### Robotics and AI
##### Beauty
##### Fun
##### Importance
##### Challenges
#### The beauty and the fun
AI and robotics are the most beautiful disciplines – it is mankind’s attempt at creating artificial creatures that think and act like us.
And it is fun to see robots go!
#### The importance and the challenges
AI and robotics will change our world. Everybody should understand the possibilities, the current status and how much is left to do.
### Learning experiences
Although we design, develop and distribute hardware, software and pedagogical materials for the Duckietown platform, this is just a means to an end: we care about the learning experience the platform enables.
#### Tangible
We learn by handling and doing. We believe that to gain real-world competence, especially in robotics, it is necessary to be able to touch a robot.
We join the maker spirit with the academic spirit: the experiences are fun, but there is also a path of academic learning.
#### Accessible
We engineer Duckietown to make the barrier of entry as small as possible: affordable, broadly available, with step-by-step instructions to go from zero to hero.
#### Inclusive
We promote a broad understanding of the effects of robotics and AI in society.
While we do not advocate for everyone to become a roboticist, we believe literacy in the topics of autonomy is needed for everyone, to develop the critical thinking skills for navigating this historical phase of integration of robots in society.
Engineers are building autonomous robots, but society at large will have to decide how much and for what to use them, as the social repercussions of these decisions will be relevant for generations to come.
---
### [AI-DO: the AI Driving Olympics](https://duckietown.com/research/ai-driving-olympics/)
**Published:** May 7, 2018
**Author:** Jacopo Tani
**Content:**
# The AI Driving Olympics (AI-DO)
###### The AI-DO are a set of competitions with the objective of evaluating the state of the art for **ML/AI for embodied intelligence**
(can AI actually DO anything?).
## Overview
Duckietown has been hosting the AI-DO competition finals twice per year, at **ICRA** (International Conference on Robotics and Automation) and **NeurIPS** (Neural Information Processing Systems Conference).
**AI-DO 2021** finals will take place in conjunction with **NeurIPS 2021**. There are **three leagues:**
- The **Urban Driving League** uses the Duckietown Platform. Read on to know all about it.
- The **Advanced Perception League** uses the [nuScenes](https://www.nuscenes.org/) dataset/challenges and is organized by [Motional](http://www.motional.com).
- The **Racing League** uses AWS Deepracer and is organized by AWS. To join the racing league, continuing reading [here](https://www.aicrowd.com/challenges/neurips-2021-aws-deepracer-ai-driving-olympics-challenge).
[ Participate in the AI-DO ](#aido-start)
### AI-DO 2021 sponsors
### AI Driving Olympics 2021 @ NeurIPS
### The voice of the experts
*"It is great to see Duckietown host the AI Driving Olympics at ICRA and NIPS. What a fun way to demonstrate the real challenges in building and deploying self driving cars!"*
John Leonard, Prof.
Massachusetts Institute of Technology
*"Understanding the behavior of AVs is pivotal to assess their riskiness: Swiss Re enthusiastically supports the Duckietown and AI Driving Olympics initiatives."*
Luigi Di Lillo, Dr.
Swiss Re
*"The AI Driving Olympics offer a glimpse of the challenges of creating self-driving cars. A playful but rigorous competition on a smaller and safer testbed is the best way to develop the creativity needed to make progress in this field."*
Emilio Frazzoli, Prof.
ETH Zurich / Motional
*"The AI Driving Olympics is a great way to push the limits of deep learning on physically embodied systems."*
Joshua Bengio, Prof.
University of Monreal
### Get started with the AI-DO webinars
**Introduction to AI-DO 5**
**ROS and Duckietown baselines**
**Local development on Duckiebot (DB19)**
**Reinforcement learning baseline**
**Imitation learning baseline**
### Urban Driving League
The urban driving league is based on the [Duckietown platform](https://duckietown.com/), and includes a series of tasks of increasing complexity aiming at solving precision driving and safety-critical challenges.
The competition has two stages: participants access the finals by obtaining high placements in the simulated challenges leaderboards, and winners are determined based on experimental evaluations performed in remote Duckietown Autolabs.
[ Check ongoing edition ](https://www.duckietown.com/archives/82927)
[ Check previous edition ](https://www.duckietown.com/archives/64683)
#### The Challenges
The challenges range from single robot tasks such as lane following (LF) on road-loop map to complex multi-robot behaviors such as lane following with intersection and other vehicles in the presence of pedestrians (LFIVP).
Challenge complexity evolves across several dimensions, the:
- **road complexity**: intersections, traffic signs or traffic lights,
- **number of vehicles**: same or opposite lane, passively or actively controlled
- **presence of pedestrians** (duckies) to plan around.
Multi-vehicle challenges support “multiplayer” mode, where **your agent is embodied in multiple vehicles**.
Each AI-DO edition focuses on a subset of challenges. You can find information on the ongoing AI-DO here, and all the challenges on the [challenges server](https://challenges.duckietown.org/v4/humans/challenges).
LF: Lane following challenge
LFP: Lane following with pedestrians challenge
LFV: Lane following with vehicles challenge
LFI-TL: Lane following with intersections and traffic lights
LFVI: Lane following with vehicles and intersections challenge
#### Everyone can compete
Participants will not need to be physically present at any stage of the competition.
Competitors submit their solutions to specific challenges in the form of agents packaged as a Docker container.
The agents are evaluated first in simulation (remotely, locally and/or in the cloud), and then the same code is tested on physical robots in a Duckietown Autolab.
The technical infrastructure supporting the AI-DO Urban Driving League is described [here](https://www.duckietown.com/archives/59214).
#### Resources
We provide tools for competitors to use in the form of simulators, logs, code templates, baseline implementations and low-cost access to robotic hardware.
- [ Challenges ](https://docs.duckietown.org/daffy/AIDO/draft/aido_rules.html)
- [ Simulators ](https://docs.duckietown.org/daffy/AIDO/draft/dt_simulator.html)
- [ Logs ](http://logs.duckietown.org/)
- [ Code templates ](https://docs.duckietown.org/daffy/AIDO/draft/embodied.html)
- [ Baseline implementations ](https://docs.duckietown.org/daffy/AIDO/draft/embodied_strategies.html)
- [ Accessible hardware ](https://get.duckietown.com/collections/ai-do-kits)
#### Challenge server
The [challenges server](https://challenges.duckietown.org/v4/) allows to control one’s submissions, and to see the leaderboards.
### Get Started with the AI-Driving Olympics
- Check the rules of the **[ongoing edition](https://www.duckietown.com/archives/82927)** ([AI-DO 2021 Urban League Finalists](https://www.duckietown.com/archives/85031)).
- The **[Quickstart Guide](http://docs.duckietown.org/daffy/AIDO/out/quickstart.html)**: a handy chapter of the AI-DO book
- Join the **[Slack community](https://join.slack.com/t/duckietown/shared_invite/enQtNTU0Njk4NzU2NTY1LWM2YzdlNmJmOTg4MzAyODc2YTI3YTc5MzE2MThkZGUwYTFkZWQ4M2ZlZGU1YTZhYjg5YTgzNDkyMzI2ZjNhZWE)** to interact with the community and get support
- [**AI-DO book**](http://docs.duckietown.org/daffy/AIDO/out/): find everything you need to know about the AI-DO
- Find the leaderboards on the [**challenges server**](https://challenges.duckietown.org/v4/): the virtual competition arena
- **[Get the AI-DO Urban league hardware kits](https://get.duckietown.com/collections/ai-do-kits/)** to deploy your solutions locally
### (Hidden) Webinars
1. Lane Following (`LF`), in which you need to **follow a lane.**
2. Lane Following with Pedestrians Vehicles (`LFP`), in which you need to avoid the **duckie-pedestrians**.
3. Lane Following with other Vehicles, multibody (`LFV_multi`), in which **your agent is embodied in multiple vehicles.**
#### AI-DO 5
##### AI-DO 5 (NeurIPS 2020) Webinar 1
- Introduction to AI-DO 5
##### AI-DO 5 (NeurIPS 2020) Webinar 2
- ROS + Duckietown baseline
##### AI-DO 5 (NeurIPS 2020) Webinar 3
- Local development on Duckiebot (DB19)
##### AI-DO 5 (NeurIPS 2020) Webinar 4
- Reinforcement learning baseline
##### AI-DO 5 (NeurIPS 2020) Webinar 5
- Imitation learning baseline
[ Sync Webinar Calendar ](https://calendar.google.com/calendar/ical/c_e5eb8m6v5f3krbkl3d00ah4at0%40group.calendar.google.com/public/basic.ics)
#### AI-DO 3 (2019)
##### AI-DO 3 (2019) Webinar 1
- Introduction to AI-DO 3
- Minimal agent template
##### AI-DO 3 (2019) Webinar 2
- ROS template
- Duckietown baseline
##### AI-DO 3 (2019) Webinar 3
- TensorFlow template
- Imitation learning from logs and using a simulator
##### AI-DO 3 (2019) Webinar 4
- PyTorch template
- Reinforcement learning baseline
#### AI-DO 3 (2019) Webinar 5
- Training in the cloud with SageMaker
#### AI-DO 3 (2019) Webinar 6
- Local development: deploying on the Duckiebot (DB18)
## (Hidden) Previous Webinars
##### Dates
##### Presenter
##### Topics
##### Recorded Video
Thurs. Oct 31
- Introduction
- Description of the challenges
- Minimal agent template
Fri. Nov 1
- Classic template (ROS)
- Duckietown baseline
Mon. Nov 4
- Tensorflow template
- IL logs
- IL simulator
Tues. Nov. 5
- Pytorch template
- RL
Wed. Nov. 6
- Training in the cloud
Thurs. Nov 14
- Running on the Duckiebot
### FAQs
**Do I need to attend the conference to compete?**
No! If you are not present at the conference where the finals will be hosted we will run your submission on your behalf.
**How do I get help?**
Join the Duckietown international [Slack community](https://join.slack.com/t/duckietown/shared_invite/enQtNTU0Njk4NzU2NTY1LTQ2MDI4MTY1OTE1YjhjMTU4YTdkMDViMzJmNmJkNGQxN2U1ZGJlZjk2NGM0M2FiODY3YmQ2MTQ3MGM2MjY1ZTI) and ask away!
**How do I get the hardware to test on a real robot?**
Specially crafted hardware kits for each challenge are available [here](https://get.duckietown.com/collections/ai-do-kits/). For any question, you can reach out to hardware@duckietown.com.
### Past Competitions
The **first edition** of the AI-DO took place in December 2018, at the **2018 Neural Information Processing Systems (NeurIPS)**, the premiere machine learning conference, in Montréal. This was the first ever competition with real robots to take place at NeurIPS. AI-DO 1 only had the Urban league, and only the Lane Following challenge. There were over 1600 submissions from 58 unique participants.
Read a [summary of the event here](https://www.duckietown.com/archives/32095).
The **second edition** of AI-DO took place at the **2019 International Conference on Robotics and Automation (ICRA)**, with finals held in Montréal, Canada, in May 2019.
AI-DO 2 comprised again only the Urban league, but additional challenges were added, such as Lane following with other vehicles (LFV) and Lane following with other Vehicles and Intersections (LFVI). The number of submissions to AI-DO 2 was similar to the number for AI-DO 1.
[Find out who won!](https://www.duckietown.com/archives/37690)
The third edition was held at NeurIPS 2019 with finals held in Vancouver, Canada. In AI-DO 3 we introduced the advanced perception and racing leagues. AI-DO 3 received over 2000 submissions across all of the leagues.
[Read all about it!](https://www.duckietown.com/archives/48581)
The **fourth edition** was scheduled for **ICRA 2020** in Paris, France, but was unfortunately **cancelled as a result of the COVID-19 outbreak.**
The **fifth edition** of AI-DO was in conjunction with **NeurIPS 2020.** Due to the COVID-19 pandemic, the conference and finals are held virtually. AI-DO 5 will comprise two leagues: Urban Driving and Advanced Perception, with novel challenges in each.
The **sixth edition** of AI-DO is in conjunction with **NeurIPS 2021.** It features three leagues: Urban Driving, Advanced Perception, and Racing. A new challenge is included in the Urban Driving league, and new Duckiebots are used.
---
### [Instagram Link Tree: explore the Duckietown world](https://duckietown.com/instagram-quick-links/)
**Published:** August 16, 2021
**Author:** Duckietown Admin
**Content:**
# Instagram Quick Links
###### Welcome to the Duckietown ecosystem - a professional robotics and AI education and training platform.
You can start using our resources today to start learning robotics and artificial intelligence (AI) like the world's best professionals.
## Recent posts
[ Visual Domain Randomization for Sim-2-Real Transfer ](https://duckietown.com/enhancing-visual-domain-randomization-for-sim2real-transfer/)
[ Safe Reinforcement Learning (RL) Thesis Project ](https://duckietown.com/safe-reinforcement-learning-rl-duckietown-thesis-project/)
[ People of Duckietown: Prof. Bruegge ](https://duckietown.com/prof-bruegge-interview-failing-successfully/)
[ MIT BCI Hackathon ](https://duckietown.com/mit-bci-hackathon/)
## Duckietown Resources
[ Learn about the Duckietown project ](https://www.duckietown.com/#learnmore)
[ Get Started (All Paths) ](https://www.duckietown.com/guides)
[ Start Learning ](/mooc)
[ Get the hardware ](https://get.duckietown.com/)
[ Research Projects ](https://duckietown.com/research-papers/)
[ Student Projects ](https://duckietown.com/projects-for-learning-robotics-and-ai/)
[ People of Duckietown ](/news/people-of-duckietown)
[ Check the Duckietown library ](https://docs.duckietown.com/daffy/)
[ Duckietown GitHub ](https://github.com/duckietown)
## Duckietown News
[](https://duckietown.com/duckie-day-2026-salmon-farming/)## [ Duckie Day 2026 brings robotics to salmon farming ](https://duckietown.com/duckie-day-2026-salmon-farming/)
[](https://duckietown.com/teaching-robot-autonomy-the-hague-university/)## [ Teaching robot autonomy at The Hague University ](https://duckietown.com/teaching-robot-autonomy-the-hague-university/)
[](https://duckietown.com/robotics-rome-cup-2026/)## [ Rome Cup 2026 ](https://duckietown.com/robotics-rome-cup-2026/)
[](https://duckietown.com/real-time-reinforcement-learning-in-duckiematrix/)## [ Real-Time Reinforcement Learning in Duckiematrix ](https://duckietown.com/real-time-reinforcement-learning-in-duckiematrix/)
[](https://duckietown.com/db21v3-j-duckiebot-upgrade-kit-is-out/)## [ DB21v3-J Duckiebot upgrade kit now available ](https://duckietown.com/db21v3-j-duckiebot-upgrade-kit-is-out/)
[](https://duckietown.com/duckiebot-db21j-now-available-pre-assembled-and-initialized/)## [ Duckiebots now available pre-assembled and initialized ](https://duckietown.com/duckiebot-db21j-now-available-pre-assembled-and-initialized/)
[](https://duckietown.com/sim-to-sim-to-real-transfer-for-small-autonomous-vehicles/)## [ Sim-to-Sim-to-Real Transfer for Small Autonomous Vehicles ](https://duckietown.com/sim-to-sim-to-real-transfer-for-small-autonomous-vehicles/)
[](https://duckietown.com/learning-robot-autonomy-with-duckiedrones/)## [ Learning robot autonomy with Duckiedrones ](https://duckietown.com/learning-robot-autonomy-with-duckiedrones/)
[](https://duckietown.com/tor-vergata-university-and-duckietown-partner-to-deliver-a-hands-on-control-systems-workshop-at-the-eu-maker-faire-rome-2025-2/)## [ Tor Vergata University and Duckietown deliver hands-on control systems workshop at EU Maker Faire Rome 2025 ](https://duckietown.com/tor-vergata-university-and-duckietown-partner-to-deliver-a-hands-on-control-systems-workshop-at-the-eu-maker-faire-rome-2025-2/)
[](https://duckietown.com/european-maker-faire-2025/)## [ Duckietown at the European Maker Faire 2025 – Rome ](https://duckietown.com/european-maker-faire-2025/)
[](https://duckietown.com/autoduck-vlm-autonomous-navigation-in-duckietown/)## [ Autoduck: VLM-based Autonomous Navigation in Duckietown ](https://duckietown.com/autoduck-vlm-autonomous-navigation-in-duckietown/)
[](https://duckietown.com/visual-control-for-autonomous-navigation-in-duckietown/)## [ Visual Control for Autonomous Navigation in Duckietown ](https://duckietown.com/visual-control-for-autonomous-navigation-in-duckietown/)
« PreviousPage1[Page2](https://duckietown.com/instagram-quick-links/2/?doing_wp_cron=1786646276.4517979621887207031250)[Page3](https://duckietown.com/instagram-quick-links/3/?doing_wp_cron=1786646276.4517979621887207031250)…[Page13](https://duckietown.com/instagram-quick-links/13/?doing_wp_cron=1786646276.4517979621887207031250)[Next »](https://duckietown.com/instagram-quick-links/2/)
---
### [Request information](https://duckietown.com/request-information/)
**Published:** December 2, 2023
**Author:** Jacopo Tani
**Content:**
# Request information
###### Would you like additional information regarding Duckietown products, services or use cases? Reach out through the form below.
---
### [Join the team](https://duckietown.com/join-the-team/)
**Published:** September 29, 2022
**Author:** Duckietown Admin
**Content:**
# Join the Duckietown team
###### Help us bring robot autonomy education where no duckie has gone before
`; var dtJobListingFmt = ` {content} {actions} `; (function () { // Localize jQuery variable var jQuery; /******** Our main function ********/ function main() { jQuery(document).ready(function ($) { // We can use jQuery here // set a default source code if (typeof ht_settings.src_code === "undefined" || ht_settings.src_code === null) { ht_settings.src_code = "standard"; } if ( typeof ht_settings.open_jobs_in_new_tab === "undefined" || ht_settings.open_jobs_in_new_tab === null ) { ht_settings.open_jobs_in_new_tab = false; } var container = $("#hiringthing-jobs"); var spinner = $(''); container.html(spinner); var site_url = ht_settings.site_url; if (typeof site_url === "string") { site_url = [site_url]; } var promises = []; $.each(site_url, function (idx, site_url) { promises.push( $.ajax({ url: "https://" + site_url + ".applicant-tracking.com/api/widget_jobs?src=" + ht_settings.src_code + "&callback=?", type: "GET", dataType: "json", }) ); }); $.when .apply($, promises) .done(function (response) { var jobs = []; if (promises.length == 1) { Array.prototype.push.apply(jobs, response); } else { $.each(arguments, function (idx, response) { Array.prototype.push.apply(jobs, response[0]); }); } var str = ""; for (var i = 0; i ' + jobs[i].title + ""; if (jobs[i].location) { jobListing += '' + jobs[i].location + "
"; } //str += '' + jobs[i].summary + "
"; /*str += ahref_start + 'aria-label="Apply now to ' + jobs[i].title + '"' + ' class="ht-apply-link">Apply Now';*/ let btn = dtBtn.format({color: 'yellow', text: 'Apply now!', href: jobs[i].apply_url, scale: 0.8, target: ahref_target}); str += dtJobListingFmt.format({content: jobListing, actions: btn}); //str += dtBtn.format({color: 'yellow', text: 'Apply now!', href: jobs[i].apply_url, scale: 0.8, target: ahref_target}); } if (str == "") { str = 'We have no open positions at this time.
'; } container.html(str); }) .fail(function () { container.html( "Account not found.
Please configure 'site_url' to match your Applicant Tracking account domain. " ); }); }); } /******** Load jQuery if not present *********/ if (window.jQuery === undefined || window.jQuery.fn.jquery !== "3.3.1") { /******** Called once jQuery has loaded ******/ var onloadHandler = function () { // Restore $ and window.jQuery to their previous values and store the // new jQuery in our local jQuery variable jQuery = window.jQuery.noConflict(true); // Call our main function main(); }; var script_tag = document.createElement("script"); script_tag.setAttribute("type", "text/javascript"); script_tag.setAttribute("src", "https://code.jquery.com/jquery-3.3.1.min.js"); if (script_tag.readyState) { script_tag.onreadystatechange = function () { // For old versions of IE if (this.readyState === "complete" || this.readyState === "loaded") { onloadHandler(); } }; } else { // Other browsers script_tag.onload = onloadHandler; } // Try to find the head, otherwise default to the documentElement (document.getElementsByTagName("head")[0] || document.documentElement).appendChild(script_tag); } else { // The jQuery version on the window is the one we want to use jQuery = window.jQuery; main(); } })(); // We call our anonymous function immediately
## Reach out to the team
For any questions, comments, doubts, spontaneous applications, or words of encouragement, do not hesitate to reach out to us!
[ Contact us ](/contact)
---
### [A brief history of Duckietown](https://duckietown.com/history/)
**Published:** May 7, 2018
**Author:** Jacopo Tani
**Content:**
## The history of Duckietown
The Duckietown project was conceived in 2016 as a graduate class at the Massachusetts Institute of Technology (MIT).
The goal was to build a platform that was small-scale and accessible yet still preserved the scientific challenges inherent in a full-scale real autonomous robot platform.
### The very first Duckietown lecture at MIT in 2016
All the course lectures are available [here](https://vimeo.com/showcase/10078155).
### The first demo
The first class had 24 students, over 15 postdocs, and 5 professors involved in the initial development.
The course culminated with a year-end demo [showcasing the capabilities](https://vimeo.com/showcase/10079980) of the platform in a full-sized hockey rink in Cambridge, MA.
There were over 3000 visitors.
### Going global
After the 2016 class, many of the key organizers left MIT for other opportunities.
In the meantime, all of the pieces of the experience (the slides, the demos, the platform, the software) were made openly available and other institutions began to take interest.
First adopters, precursors of many more universities, included NCTU in Taiwan, Tsinghua University in China, and Rensselaer Polytechnic Institute in the United States.
### The joint class of 2017
In the Fall of 2017, Liam Paull (Montréal), Andrea Censi and Jacopo Tani (ETHZ), Matthew Walter (TTIC), and Hsueh-Cheng Wang (NCTU) decided to teach an officially coordinated edition of the class. In this version, students from classes at ETH Zürich, Université de Montréal, TTI Chicago, and NCTU worked collaboratively in groups that spanned continents.
The result was a global demo that showcased the students’ achievements to the general public.
### The 2018 Kickstarter
The requests from universities, companies, and makedemics around the world to use the platform started increasing dramatically.
To support this growth, the Duckietown platform had to become more affordable, easier to obtain, and of higher-quality, so as to provide more learning experiences and opportunities for performing cutting-edge research while creating the least possible entropy as the user base increased in size.
To achieve this objective we launched a successful [Kickstarter campaign](https://www.kickstarter.com/projects/163162211/duckietown-a-playful-road-to-learning-robotics-and) and created an easier way for people to [acquire the hardware](https://get.duckietown.com/).
### The AI Driving Olympics
The Duckietown Foundation debuted the [AI Driving Olympics](/research/AI-Driving-olympics) (AI-DO), a competition focused on AI for self-driving cars, in December 2018 at the premiere machine learning conference: Neural Information Processing Systems conference (NeurIPS) in Montreal.
It was the first competition to ever take place at a machine-learning conference with real robots.
### Science Museum of London Exhibition
Duckietown is a trailblazer in providing accessible solutions for teaching and learning state-of-the-art autonomy, helping in the dissemination of the science and technology of modern robotics.
In 2019, the Science Museum of London picks up on the project and includes Duckietown in their “[Driverless: Who is in control?](https://www.sciencemuseumgroup.org.uk/our-services/partner-with-us/touring-exhibitions/driverless-who-is-in-control/)” exhibition.
(Pictures are courtesy of the Board of the Science museum of London.)
#### Duckietown's historical impact: the permanent collection
In August 2023, several Duckietown objects officially joined the Science Museum Group permanent Collection: the United Kingdom’s national archive of science, technology, engineering, and medicine.
This will “ensure that the items – as rare and representative objects – are acquired, conserved, preserved and stored in order that they may be accessible to current and future generations for interpretation, loans to other institutions, research, education, and sometimes display in temporary or permanent exhibitions.”
The permanent collection can be browsed from [here](https://collection.sciencemuseumgroup.org.uk/ "Science Museum Group Duckietown archive"):
2023-216
E2019.0205.1
Duckiebot – small autonomous robot from Duckietown Project, 2018-2019
2023-222
E2019.0205.2
Traffic light kit from Duckietown Project, 2018-2019
2023-502
E2019.0205.3
City expansion pack from Duckietown Project, 2018-2019
### Edtech awards
The Duckietown platform is recognized by the [EdTech Awards](https://www.edtechdigest.com/the-edtech-awards/), becoming a finalist: - (2020) as a product or service setting a trend in education technologies
- (2020) as higher education solution
- (2020) as robotics (for learning, education) solution
- (2021) as an online course, MOOC solution
### "Self-Driving Cars with Duckietown" massive open online course
The Duckietown Foundation supported the creation of the world’s first robot autonomy massive open online course with hardware.
[Self-Driving Cars with Duckietown](/mooc) is hosted on the edX platform, and the first cohort in 2021 counted over 7000 learners from over 170 countries.
The course development was spearheaded by ETH Zurich, and developed in collaboration with the University of Montreal and the Toyota Technological Institute at Chicago.
---