Anatidaephilia: centralized city-based SLAM (cSLAM)

Posted on April 26, 2024 | by Duckietown Admin

Anatidaephilia: centralized city-based SLAM (cSLAM)

Project Resources

cSLAM Project Description

Project cSLAM – Simultaneous Localization and Mapping (SLAM) is a successful approach for robots to estimate their position and orientation in the world they operate in, while at the same time creating a representation of their surroundings.

This project, centralized SLAM (or cSLAM), enables a Duckiebot to localize itself, while the watchtowers and Duckiebots work together to build a map of the city. The task is achieved by using the camera of the Duckiebot, together with watchtowers located along the path, to detect AprilTags attached to the tiles, the traffic signs, and the Duckiebot itself.

P. S. Anatidaephilia, is Latin for loving, and being addicted to, the idea that somewhere, somehow, a duck is watching you.

Project Highlights

Here is a visual tour of the work of the authors.

Check out the documents for more details!

Duckietown cSLAM algorithm physical architecture for localizing Duckiebots — cSLAM algorithm physical architecture for localizing Duckiebots. Watchtowers are traffic lights without lights, and are used to transform Duckietown in Autolabs.

Duckietown cSLAM logical architecture for merging sensor measurements from robots and the city.

Duckietown cSLAM graph optimizer representation — AprilTag detections from Watchtowers and Duckiebots are harmonized by solving a minimization problem.

RViz visualization of Autolab virtual twin in Duckietown cSLAM project — RViz reconstruction of experimental localization outcomes.

cSLAM Autolab watchtowers network diagnostic

cSLAM Project Results

(Turn on the sound for best experience!)

This work developed into a paper, check the article here.

Project Authors

Rohit Suri is a former Duckietown student, now a Roboticist at Venti Technologies in Singapore.

Aleksandar Petrov is a former Duckietown student, now a Ph. D. student at the University of Oxford.

Amaury Camus is a former Duckietown student, now a Lead Robotics engineer at Hydromea SA in Switzerland.

Francesco Milano is a former Duckietown student, now a Ph. D. student at ETH Zurich in Switzerland.

Benson Kuan is a former Duckietown student, now a Senior Robotics Research Engineer at DSO National Laboratories in Singapore.

Learn more

Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.

It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.

Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm

Posted on April 4, 2024 | by Duckietown Admin

General Information

Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm
Ambedkar Dukkipati, Rajarshi Banerjee, Ranga Shaarad Ayyagari, Dhaval Parmar Udaybhai
Indian Institute of Science
A. Dukkipati, R. Banerjee, R. S. Ayyagari and D. P. Udaybhai, "Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm," 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 2022, pp. 2483-2489, doi: 10.1109/IROS47612.2022.9981607.

Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm

Reinforcement learning (RL) is a rising star approach for developing autonomous robot agents. The essence of RL is training agents based on policies that reward desirable outcomes, which leads to increased adaptability to operational scenarios. Through iterations, robots refine their decision-making, optimizing actions based on rewards and penalties. This method provides robots with the flexibility to handle unpredictable situations, enhancing their efficiency and effectiveness in real-world tasks. To learn about RL with Duckietown, check out the resources below.

Abstract

Solving complex problems using reinforcement learning necessitates breaking down the problem into manageable tasks, and learning policies to solve these tasks. These policies, in turn, have to be controlled by a master policy that takes high-level decisions. Hence learning policies involves hierarchical decision structures. However, training such methods in practice may lead to poor generalization, with either sub-policies executing actions for too few time steps or devolving into a single policy altogether. In our work, we introduce an alternative approach to learn such skills sequentially without using an overarching hierarchical policy. We propose this method in the context of environments where a major component of the objective of a learning agent is to prolong the episode for as long as possible. We refer to our proposed method as Sequential Soft Option Critic. We demonstrate the utility of our approach on navigation and goal-based tasks in a flexible simulated 3D navigation environment that we have developed. We also show that our method outperforms prior methods such as Soft Actor-Critic and Soft Option Critic on various environments, including the Atari River Raid environment and the Gym-Duckietown self-driving car simulator.

Highlights

Here is a visual tour of the work of the authors.

For all the details, check out the paper link!

(a) An illustration of policies learned in our approach. (b) A representation of the state space partitioned by the termination functions of the options. Each oval corresponds to the set of states classified as non-termination states by the corresponding nested termination function. — Fig. 1. (a) An illustration of policies learned in our approach. Different policies learn different skills out of necessity to traverse the environment and not terminate the episode. The trajectory spawned by the sequence of policies that correctly learns to avoid terminating the episode is shown in green. The policy π1 is used when the agent is in the middle of the corridor. When the end of the corridor is reached, π2 is selected to take a turn and avoid a collision. (b) A representation of the state space partitioned by the termination functions of the options. Each oval corresponds to the set of states classified as non-termination states by the corresponding nested termination function.

Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithm image — Fig. 2. (a) & (b) The input to the neural network in our 3D navigation environment. These images obtained from the simulation are scaled (and also transformed to grayscale in the case of (a)) and appended with K previous images, and sent to the agent as input. (c) A view of the Duckietown environment.

Autonomous robot Navigation performance results obtained on various environments: SAC, SOC, SAC with HER — Fig. 3. The results obtained on various environments.

Outputs of the three learned policies when fed the corresponding input shown on the left. πω1 : orange, πω2 : blue, πω3 : green. Each policy outputs a Gaussian distribution, the active policy is the one filled with color. A positive value for the output action corresponds to turning right, and a negative value indicates a left turn. — Fig. 4. Outputs of the three learned policies when fed the corresponding input shown on the left. πω_1: orange, πω_2 : blue, πω_3 : green. Each policy outputs a Gaussian distribution, the active policy is the one filled with color. A positive value for the output action corresponds to turning right, and a negative value indicates a left turn.

Conclusion

In this paper, the authors proposed an algorithm called “Sequential Soft Option Critic” that allows adding new skills dynamically without the need for a higher-level master policy. This can be applicable to environments where a primary component of the objective is to prolong the episode. We show that this algorithm can be used to effectively incorporate diverse skills into an overall skill set, and it outperforms prior methods in several environments.

Learn more

Duckietown is a platform for creating and disseminating robotics and AI learning experiences.

Duckietown is modular, customizable and state-of-the-art. It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.

Duckiedrone: how to fly a Raspberry Pi-based autonomous quadcopter

Posted on October 16, 2023 | by Duckietown Admin

Boston, 20 October 2023: Duckietown Sky and the Duckietown drone, a Raspberry Pi-based autonomous quadcopter, are discussed with the Aerial ROS community, a group of experts working to define the future of software architectures for quadcopters.

Quick links

Learning robot autonomy by flying with Duckietown Sky

Following an invitation to the Aerial ROS workgroup community meeting, Duckietown staff was delighted to present the Duckietown Sky initiative, the current Duckiedrone design, a DIY Raspberry Pi-based autonomous quadcopter, and future plans for both hardware and courseware development.

The goal of the ROS (Robotic Operating System) aerial robotics working group is to gather drone enthusiasts within the ROS community and facilitate the sharing of ideas and discussion of issues regarding autonomous robotic platforms operating in the air.

Duckietown Sky, a National Science Foundation-funded educational effort in collaboration with Brown University started in 2019, is an integral component of the Duckietown education vision, representing the commitment to fostering robot autonomy education in all its forms. Beyond self-driving cars (Duckiebots) and smart cities (Duckietowns), Duckietown highlights what is common despite the different applications of robot autonomy. From ground to sky, whether it drives, flies, or blinks, Duckietown is a platform to learn, explore and innovate when it comes to robot autonomy.

With the focus on quadcopters, Duckietown Sky offers MOOC-style learning experiences tailored for undergraduate and senior high school students. Flight is exciting!

The program’s design criteria revolves around achieving state-of-the-art autonomy ground-up using off-the-shelf components, with a Raspberry Pi as core computational unit, for its wide-spread applications and large community. Duckiedrones, now at the second hardware design iteration moving towards the third, aim to provide students with hands-on learning experiences covering from the basics, such as soldering, to pretty advanced algorithmic cornerstones of autonomy such an UKFs (Unscented Kalman Filters) and SLAM (Simultaneous Localization and Mapping).

Aspiring engineers should however be prepared for preliminary requirements like soldering, have access to a laptop or base station, and an internet connection for setting the working environment up.

From a box of parts to a Raspberry Pi-based autonomous quadcopter

The Duckietown Sky experience is an exciting journey that begins with a simple box of parts and culminates in the creation of an autonomously flying drone. In the Duckietown spirit of democratizing access to the science and technology of autonomy through accessible platforms, the happy-yellow Duckiebox includes almost everything needed to get flying.

We encourage instructors, students and practitioners to check the development roadmaps for both our hardware design and courseware, outlined in our presentation, and reaching out without hesitation to provide comments or feedback!

Building on the extensive experience of the Duckietown team in massive open online courses (check out the Self-Driving Cars with Duckietown MOOC: the world’s first robot autonomy MOOC with hardware), we look to prepare a series of short online courses. These courses will be led by Professors from Brown, as well as other universities, and will provide an ever broader audience with the opportunity to explore the fascinating world of robot autonomy: from the science and technology to the tools and workflows, to real-world applications presented by industry and academic leaders.

Want to try the Duckietown Sky experience yourself, build a DYI Raspberry Pi-based autonomous quadcopter, or teach an aerial autonomy class at your high school or at university level? Follow the steps below to begin now.

Learn more about Duckietown

Duckietown enables state-of-the-art robotics and AI learning experiences. It is designed to help teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of human knowledge.

Ubuntu laptop terminal interface with hands operating keyboard, Duckiebot and duckies out of focus in foreground

“Self-Driving Cars with Duckietown” MOOC starting soon

Posted on March 4, 2021 | by Duckietown Admin

Join the first hardware based MOOC about autonomy on edX!

Are you curious about robotics, self-driving cars, and want an opportunity to build and program your own? Set to start on March 22nd, 2020, “Self-Driving Cars with Duckietown” is a hands-on introduction to vehicle autonomy, and the first ever self-driving cars MOOC with a hardware track!

Designed for university-level students and professionals, this course is brought to you by the Swiss Federal Institute of Technology in Zurich (ETHZ), in collaboration with the University of Montreal, the Duckietown Foundation, and the Toyota Technological Institute at Chicago.

Learning autonomy requires a fundamentally different approach when compared to other computer science and engineering disciplines. Autonomy is inherently multi-disciplinary, and mastering it requires expertise in domains ranging from fundamental mathematics to practical machine-learning skills.

This course will explore the theory and implementation of model- and data-driven approaches for making a model self-driving car drive autonomously in an urban environment, while detecting and avoiding pedestrians (rubber duckies)!

In this course you will learn, hands-on, introductory elements of:

computer vision
robot operations
ROS, Docker, Python, Ubuntu
autonomous behaviors
modelling and control
localization
planning
object detection and avoidance
reinforcement learning.

The Duckietown robotic ecosystem was created at the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) in 2016 and is now used in over 90 universities worldwide.

“The Duckietown educational platform provides a hands-on, scaled down, accessible version of real world autonomous systems.” said Emilio Frazzoli, Professor of Dynamic Systems and Control, ETH Zurich, “Integrating NVIDIA’s Jetson Nano power in Duckietown enables unprecedented access to state-of-the-art compute solutions for learning autonomy.”

This massive online open course will be have a hands-on learning approach using, for the hardware track, real robots. You will learn how autonomous vehicles make their own decisions, going from theory to implementation, deployment in simulation as well as on the new NVIDIA Jetson Nano powered Duckiebots.

“The new NVIDIA Jetson Nano 2GB is the ultimate starter AI computer for educators and students to teach and learn AI at an incredibly affordable price.” said Deepu Talla, Vice President and General Manager of Edge Computing at NVIDIA. “Duckietown and its edX MOOC are leveraging Jetson to take hands-on experimentation and understanding of AI and autonomous machines to the next level.”

The Duckiebot MOOC Founder’s edition kits are available worldwide, and thanks to OKdo, are now available with free shipping in the United States and in Asia!

“I’m thrilled that ETH, with UMontreal, the Duckietown Foundation, and the Toyota Technological Institute in Chicago, are collaborating to bring this course in self-driving cars and robotics to the 35 million learners on edX. This emerging technology has the potential to completely change the way we live and travel, and the course provides a unique opportunity to get in on the ground floor of understanding and using the technology powering autonomous vehicles,” said Anant Agarwal, edX CEO and Founder, and MIT Professor.

Enroll now and don’t miss the chance to join in the first vehicle autonomy MOOC with hands-on learning!

The “Self-Driving cars with Duckietown” Massive Open Online Course on edX

Posted on November 16, 2020 | by Jacopo Tani

"Self-Driving Cars with Duckietown" hands-on MOOC on edX

We are launching a massive open online course (MOOC): “Self-Driving Cars with Duckietown” on edX, and it is free to attend!

This course is made possible thanks to the support of the Swiss Federal Institute of Technology in Zurich (ETHZ), in collaboration with the University of Montreal, the Duckietown Foundation, and the Toyota Technological Institute at Chicago.

This course combines remote and hands-on learning with real-world robots. It is offered on edX, the trusted platform for learning, and it is now open for enrollment.

Learning activities will support the use of Jetson Nano equipped Duckiebots, powered by NVIDIA.

Learning autonomy

Participants will engage in software and hardware hands-on learning experiences, with focus on overcoming the challenges of deploying autonomous robots in the real world.

This course will explore the theory and implementation of model- and data-driven approaches for making a model self-driving car drive autonomously in an urban environment.

Pedestrian detection

MOOC Factsheet

Prerequisites

What you will learn

Why Self-driving cars with Duckietown?

Teaching autonomy requires a fundamentally different approach when compared to other computer science and engineering disciplines, because it is multi-disciplinary. Mastering it requires expertise in domains ranging from fundamental mathematics to practical machine-learning skills.

Robot Perception

Robots operate in the real world, and theory and practice often do not play well together. There are many hardware platforms and software tools, each with its own strengths and weaknesses. It is not always clear what tools are worth investing time in mastering, and how these skills will generalize to different platforms.

Duckiebot Detection

Learning through challenges

Progressing through behaviors of increasing complexity, participants uncover concepts and tools that address the limitations of previous approaches. This allows to get Duckiebots to actually do things, while gradually re-iterating concepts through various technical frameworks. Simulation and real-world experiments will be performed using a Python, ROS, and Docker based software stack.

Robot Planning

(Hidden) This line and everything under this line are hidden

This course combines remote and hands-on learning with real-world robots.

It is offered on edX, the trusted platform for learning, and it is now open for enrollment.

Learning activities will support the use of NVIDIA Jetson Nano powered Duckiebots.

STEM Intensive Learning with Prof. Krinkin

Posted on November 18, 2019 | by Duckietown Admin | 1 Comment

In the world of engineering education, there are many excellent courses, but often the curriculum has one serious drawback – the lack of good connectivity between different topics. Over in Saint Petersburg, Russia, Kirill Krinkin from SPbETU and JetBrains Research has been using Duckietown to address this problem through an intensive STEM winter course.

STEM Intensive Learning Approach

by Kirill Krinkin

The first part of the school program was a week of classes in the base topic areas which were chosen to complement each other and help students see the connection between seemingly different things – mathematics, electronics and programming.

Of course, the main goal of the program was to give students the opportunity to put their new found knowledge into practice themselves.

Duckietown was the perfect fit for our course because it offered a hands-on learning experience for all of our main topics areas, and once we covered those subject in the first lessons, we challenged the students with much more complex tasks – in the form of projects – in the second half of the course. It made for an exciting and engaging curriculum because students could address a problem, write a program to solve it, and then immediately launch it on a real robot.

The main advantage of Duckietown compared to many other platforms is that there is a very small learning curve: people who knew nothing about programming and robotics started working on projects after only a few days!

Overview of the course

Part 1 – Main Topic Areas

Subject 1: Linear Algebra

Students spent one day studying vectors and matrices, systems of linear equations, etc. Practical tasks were built in an interactive mode: the proposed tasks were solved individually, and the teacher and other students gave comments and tips.

Subject 2: Electricity and Simple Circuits

Students studied the basics of electrodynamics: voltage, current, resistance, Ohm’s law and Kirchhoff’s laws. Practical tasks were partially done in the electric circuits simulator or performed on the board, but more time was devoted to building real circuits, such as logic circuits, oscillatory circuits, etc.

Subject 3: Computer Architecture

In a sense, a bridge connecting physics and programming. Students studied the fundamental basis, the significance of which is more theoretical than practical. As a practice, students independently designed arithmetic-logic circuits in the simulator.

Subject 4: Programming

Python 2 was chosen as the programming language, as it is used in programming under ROS. After we taught the material and gave examples of solving problems, students were challenged with their own problems to solve, which we then evaluated.

Subject 5: ROS

Here the students started programming robots. Throughout the school day, students sat at computers, running the program code that the teacher talked about. They were able to independently launch the basic units of ROS, and also get acquainted with the Duckietown project. At the end of this day, students were ready to begin the design part of the course – solving practical problems.

Part 2 – Projects

1. Calibration of colors

Duckiebots needs to calibrate the camera when lighting conditions change, so this project focussed on the task of automatic calibration. The problem is that color ranges are very sensitive to light. Participants implemented a utility that would highlight the desired colors on the frame (red, white and yellow) and build ranges for each of the colors in HSV format.

2. Duck Taxi

The idea of this project was that Duckiebot could stop near some object, pick it up and then continue along, following a certain route. Of course, a bright yellow Duckie was the chosen passenger. The participants divided this task into two: detection and movement along the graph.

drive while Duckie is not detected

Duckie identified as a yellow spot with an orange triangle 🙂

Building a route according to the road graph and destination point

3. Building a road map

The goal of this project was to build a road map without providing a priori environmental data for the Duckiebot, relying solely on camera data. Here’s the working scheme of the algorithm developed by the participants:

4. The patrol car

This project was invented by the students themselves. They offered to teach one Duckiebot, the “patrol”, to find, follow, and stop an “intruding” Duckiebot. The students used ArUco markers to identify the Intruder on the road as they are easy to work with and they allow you to determine the orientation and distance of the marker. Next, the team changed the state machine of the Patrol Duckiebot so that when approaching the stop-line the bot would continue through the intersection without stopping. Finally, the team was able to get the Patrol Duckiebot to stop the Intruder bot by connecting via SSH and turning it off. The algorithm of the patrol robot can be represented as the following scheme:

Summary

Students walked away from our STEM intensive learning program with the foundations of autonomous driving, from the theoretical math and physics behind the programming and circuitry to the complex challenges of navigating through a city. We were successful in remaining accessible to beginners in a particular area, but also providing materials for repetition and consolidation to experienced students. Duckietown is an excellent resource for bringing education to life.

After our course ended students were asked about their experience. 100% of them said that the program exceed their expectations. We can certainly say that the Duckietown platform played a pivotal role in our success.

Duckietown: An open, inexpensive and flexible platform for autonomy education and research

Posted on July 29, 2018 | by Liam Paull

Title: Duckietown: An open, inexpensive and flexible platform for autonomy education and research
Authors: Liam Paull; Jacopo Tani; Heejin Ahn; Javier Alonso-Mora; Luca Carlone; Michal Cap; Yu Fan Chen; Changhyun Choi; Jeff Dusek; Yajun Fang; Daniel Hoehener; Shih-Yuan Liu; Michael Novitzky; Igor Franzoni Okuyama; Jason Pazis; Guy Rosman; Valerio Varricchio; Hsueh-Cheng Wang; Dmitry Yershov; Hang Zhao; Michael Benjamin; Christopher Carr; Maria Zuber; Sertac Karaman; Emilio Frazzoli; Domitilla Del Vecchio; Daniela Rus; Jonathan How; John Leonard; Andrea Censi
Published: 2017 IEEE International Conference on Robotics and Automation (ICRA)

Duckietown: An open, inexpensive and flexible platform for autonomy education and research

Duckietown is an open, inexpensive and flexible platform for autonomy education and research. The platform comprises small autonomous vehicles (“Duckiebots”) built from off-the-shelf components, and cities (“Duckietowns”) complete with roads, signage, traffic lights, obstacles, and citizens (duckies) in need of transportation. The Duckietown platform offers a wide range of functionalities at a low cost. Duckiebots sense the world with only one monocular camera and perform all processing onboard with a Raspberry Pi 2, yet are able to: follow lanes while avoiding obstacles, pedestrians (duckies) and other Duckiebots, localize within a global map, navigate a city, and coordinate with other Duckiebots to avoid collisions. Duckietown is a useful tool since educators and researchers can save money and time by not having to develop all of the necessary supporting infrastructure and capabilities. All materials are available as open source, and the hope is that others in the community will adopt the platform for education and research.

Category: education