Implementing reinforcement learning, in this project faced a number of challeneges summarized below – 

• Transfer from Simulation to Reality (Sim2Real): Models trained in simulations often encountered difficulties when applied to real-world Duckiebots, requiring adjustments for accurate and stable performance.
• Computational Constraints: Limited processing power on the Duckiebots made it challenging to run complex RL models and object detection algorithms simultaneously.
• Stability and Safety of Learning Models: Guaranteeing that the Duckiebot’s actions were safe and did not lead to erratic behaviors or collisions required fine-tuning and extensive testing of the RL algorithms.
• Obstacle Detection and Avoidance: Integrating YOLO v5 for obstacle detection posed challenges in ensuring smooth integration with RL, as both systems needed to work harmoniously for obstacle avoidance.

These challenges were addressed through algorithm optimization, iterative model testing, and adjustments to the hyperparameters.

Implementing smart lighting in Duckietown to improve autonomous driving during day and night cycles presents several challenges. Here are a few examples: 

Hardware modifications: while Duckiebots are equipped with controllable LEDs, city infrastructure does not possess lighting capabilities out of the box. The first step is integrating light sources in the design of Duckietown’s city infrastructure.

Variable lighting conditions: Duckiebots, which in this project rely uniquely on vision in their autonomy pipeline, must adapt to changing lighting conditions such as full darkness, sunrise, sunset, and artificial lighting, which impacts camera vision and lane detection accuracy.

Decentralized control: managing street lighting in a decentralized way across Duckietown ensures that each area adapts to its local lighting needs, compensating for example for the presence of passing Duckiebots with their own lights on. Join control algorithms including both city infrastructure and vehicle lighting intensity add complexity to the system’s design and coordination.

Scalability: the street lighting system must be scalable across the entire city, requiring a design that can be expanded without significant complications.

Safe and reliable operation: the system needs to be safe, adapting to issues such as occasional watchtower lighting source failure, while ensuring consistent lane-following performance.

Some of the challenges in this intersection navigation project are:

Initial position uncertainty: Duckiebot’s starting alignment at the stop line may vary, requiring the system to handle inconsistent initial conditions.

Real-time feedback: the current system lacks real-time feedback, relying on pre-configured instructions that cannot adjust for unexpected events, such as slippage of the wheels, inconsistencies between different Duckiebots, and misalignment of road tiles (non-compliant assembly).

Processing speed: previous closed-loop solution attempts used April tags and Kalman filters – with implementations that ended up being too slow: with low update rates and delays.

Transition to lane following: ensuring a smooth handover from intersection navigation to lane following requires precise control to avoid collisions and lane invasion.

Implementing obstacle avoidance in Duckietown introduces the following challenges:

• Dynamic obstacle prediction: accurately predicting the movement of dynamic obstacles, such as other Duckiebots, to ensure effective avoidance strategies and timely responses.
• Computational complexity: managing the computational load of the trajectory solver, in “real-time” scenarios with varying obstacle configurations, while ensuring efficient performance on limited computation.
• Cost function design: creating and fine-tuning a cost function that balances lane adherence, forward motion, and obstacle avoidance, while accommodating both static and dynamic elements in a complex environment.
• Integration and testing: ensuring integration of the Obstavoid Algorithm with the Duckietown simulation framework and testing its performance in various scenarios to address potential failures and refine its robustness.

The Obstavoid Algorithm addresses these challenges by employing a time-dependent cost grid and Dijkstra’s algorithm for optimal trajectory planning, allowing for “real-time” obstacle avoidance.

Read more about how the Dijkstra’s algorithm is used in this student project titled “Goto-1: Planning with Dijkstra“.

It dynamically calculates and adjusts trajectories based on predicted obstacle movements, ensuring navigation and integration with the simulation framework.

Implementing Monocular Visual Odometry involves processing images at runtime, presents challenges that effect performance.

• Extracting and matching visual features from consecutive images is a fundamental task in monocular VO. This process can be hindered by factors such as low texture areas, motion blur, variations in lighting conditions and occlusions.
• Monocular VO systems face inherent scale ambiguity since a single camera cannot directly measure depth. The system must infer scale from visual features, which can be error-prone and less accurate in the absence of depth cues.
• Running VO algorithms requires significant computational resources, particularly when processing high-resolution images at a high frequency. The Raspberry Pi used in the Duckiebot has limited processing power and memory, which contrians the performance of the visual odometry pipeline (the newer Duckiebots, DB21J uses Jetson Nano for computation.)
• Monocular VO systems, as all odometry systems relying on dead-recokning models, are susceptible to long-term drift and divergence due to cumulative errors in feature tracking and pose estimation.

This project addresses visual odometry challenges by implementing robust feature extraction and matching algorithms (ORB by default) and optimizing parameters to handle dynamic environments and computational constraints. Moreover, it integrates visual odometry with existing Duckiebot autonomy pipeline, leveraging the finite state machine for accurate pose estimation and navigation.

The new planning capailities of Duckiebots enable autonomous navigation building on pre-existing functionalities, such as “lane following”, “intersection detection and identification”, and “intersection navigation” (we are operating in a scenario with only one agent on the map, so coordination and obstacle avoidance are not central to this progect).

Lane following in Duckietown is mainly vision-based, and as such suffers from the typical challenges of vision in robotics: motion blur, occlusions, sensitivity to environmental lighting conditions and “slow” sampling.

Intersection detection in Duckietown relies on the identification of the red lines on the road layer. Identification of the type of intersection, and relative location of the Duckiebot with respect to it, is instead achieved through the detection and interpretation of fiducial markers, appropriately specified and located on the map. In the case of Duckietown, April Tags (ATs) are used. Each AT, in addition to providing the necessary information regarding the type of intersection (3- or 4-way) and the position of the Duckiebot with respect to the intersection, is mapped to a unique ID in the Duckietown traffic sign database. 

These traffic signs IDs can be used to unamiguosly define the graph of the city roads. Based on this, and leveraging the lane following pipeline state estimator, it is possible to estimate the location (with tile accuracy) of the Duckiebot with respect to a global map reference frame, hence providing the agent sufficient information to know when to stop.

After stopping at an intersection, detecting and identifying it, Duckiebots are ready to choose which direction to go next. This is where the Dijkstra planning algorithm comes into play. After the planner communicates the desired turn to take, the Duckiebot drives through the intersection, before switchng back to lane following behavior after completing the crossing. In Duckietown, we refer to the combined operation of these states as “indefinite navigation”. 

Switching between different “states” of the robot mind (lane following, intersection detection and identification, intersection navigation, and then back to lane following) requires the careful design and implementation of a “finite state machine” which, triggered by specific events, allows for the Duckiebot to transition between these states. 

Integrating a new package within the existing indefinite navigation framework can cause inconsistencies and undefined behaviors, including unreliable AT detection, lane following difficulties, and inconsistent intersection navigation.

Performance evaluation of the GOTO-1 project involved testing three implementations with ten trials each, revealing variability in success rates.

Some of the key challenges associated with vision-based object detection are the following:

Robustness across variable lighting conditions: Ensuring accurate object detection under diverse lighting is complex due to changes in object appearance (check out why in our computer vision classes). The model must handle different lighting scenarios effectively.

Balancing robustness and performance: There’s a trade-off between robustness to lighting variations and achieving high accuracy in standard operating conditions. Prioritizing one may affect the other.

Integration and real-time performance: Integrating the trained neural network (NN) model into the Duckiebot’s system is required for real-time operation, avoid lags associated with transport of images across networks. The model’s complexity therefore must align with the computational resources available. This project was executed on DB19 model Duckiebots, equipped with Raspberry Pi 3B+ and a Coral board.

Data quality and generalization: Ensuring the model generalizes well despite potential biases in the training dataset and transfer learning challenges is crucial. Proper dataset curation and validation are essential.

Some of the key challenges associated with this project are the following:

Detection Accuracy: The Duckiebot and Duckies detection systems occasionally produce false positives. Light sources from other Duckiebots or shiny objects can interfere with the LED detection, while yellow line segments can be mistaken for Duckies. Improving the reliability of detection under varying lighting conditions is essential.

Lane Following Stability: The Duckiebots sometimes become unstable while overtaking, especially when driving in the left lane. The lane-following system struggles with large lane pose angles or rapid changes in lane position, which can cause the Duckiebot to veer off the road. Enhancing the lane-following algorithm to maintain stability during lane changes is critical.

Velocity Estimation: Estimating the speed of moving Duckiebots accurately is challenging. The current position data obtained from LED detection fluctuates too much to provide a reliable velocity measurement. Developing a more robust method for estimating the velocity of other Duckiebots is needed to ensure safe and efficient overtaking.

Variable Speed Control: Implementing variable speed control during overtaking is problematic due to instability in the lane-following pipeline when speeds are dynamically adjusted. Adjusting speed based on the detected obstacle’s speed without losing lane stability is difficult, necessitating improvements in the lane control model to handle speed changes effectively.