An interview with Robert Moni
Robert is a Ph. D. student at the Budapest University of Technology and Economics.
His work focuses on deep learning and he has (co)authored papers on reinforcement learning (RL), imitation learning (IL), and sim-to-real learning using for autonomous vehicles using Duckietown.
Robert and his team won the LFV_multi hardware challenge of the 2020 AI Driving Olympics.
Today, Robert shares some of his thoughts with us!
What brought you to work on AVs?
I started my journey in the world of AV’s in 2016 when I was hired at the automotive supplier company “Continental” in Romania. In 2018 I moved to Budapest, Hungary, to join Continental’s Deep Learning Competence Center where we develop novel perception methods for AVs.
In 2019, with the support of the company, I started my Ph.D. at Budapest University of Technology and Economics on the topic “Deep Reinforcement Learning in Complex environments”.
At this time, I crossed paths with the Duckietown environment. Continental bought 12 Duckiebots and supplementary materials to build our own Duckietown environment in a lab at the university.
Tell us about you and your team
At the beginning of my Ph. D. and with the arrival of the Duckietown materials we established the “PIA” (Professional Intelligence for Automotive) project with the aim to provide education and mentorship for undergrad and master students in the field on Machine Learning and AV.
In each semester since 2019 February I managed a team of 4-6 people developing their own solutions for AI-DO challenges. I wrote a short blogpost presenting my team and our solutions submitted to AI-DO 5.
"With the arrival of the Duckietown material we established the PIA project with the aim to provide education and mentorship for undergrad and master students in the field on Machine Learning and autonomous vehicles (AV)."
What approach did you choose for AI-DO, and why?
I started to tackle the AI-DO challenges applying deep reinforcement learning (DRL) for driver policy learning and state representation learning (SRL) for sim2real transfer.
The reason for my chosen approach is my Ph. D. topic, and I plan to develop and test my hypotheses in the Duckietown environment.
What are the hardest challenges that you faced in the competition?
In the beginning, there was a simple agent training task that caused some headaches: finding a working DRL method, composing a good reward function, preprocessing the observations to reduce the search space, and fine-tuning all the parameters. All these were challenges, but well-known ones in the field.
One unexpected challenge was the continuous updates of the gym-duckietown environment. While we are thrilled that the environment gets improved by the Duckietown team, we faced occasional breakdowns in our methods when applying them to the newest releases, which caused some frustration.
The biggest headache was caused by the different setups in the training and evaluation environments: in the evaluation environment, the images are dimmed while during training they are clear. Furthermore, the real world is full of nuisances – for example lags introduced by WiFi communication, which causes different outcomes in the real environment. This challenge can be mitigated to some degree with the algorithms running directly on the Duckiebot’s hardware, and by using a more powerful onboard computer, e.g., the Jetson Nano 2GB development board.
Are you satisfied with the final outcome?
I am satisfied with the achievements of my team, which kept the resolve throughout the technical challenges faced.
I’m sure we would’ve done even better in the real-world challenge if we had seen our submission running earlier in the Autolab, so we could have adjusted our algorithms. We are going to work to bring one to our University in the next future.
What are you going to change next time?
I believe the AI-DO competition as well as the Duckietown platform would improve through more powerful hardware. I hope to see Duckiebots (DB19) upgraded to support the new Jetson Nano hardware!
(Since the date of the interview, Duckiebots model DB21 supports Jetson Nano boards)
Learn more about Duckietown
The Duckietown platform offers robotics and AI learning experiences.
Duckietown is modular, customizable and state-of-the-art. It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.
Tell us your story
Are you an instructor, learner, researcher or professional with a Duckietown story to tell? Reach out to us!