Smart Lighting: Realistic Day and Night in Duckietown

Smart Lighting: Realistic Day and Night in Duckietown

Posted on | by Duckietown Admin

Smart Lighting: Realistic Day and Night in Duckietown

Project Resources
  • Objective: City Realism: Characterising Night and Day Autonomous Driving Performance in the Duckietown Autolab
  • Approach: Developing an automated street "smart" lighting system and control models, including a PI controller.
  • Authors: David Müller
Report
Final Result
Authors

Project Highlights

Here is the output of the authors’ work on smart lighting autonomous driving.

A diagram showing the flow of nodes in the image processing pipeline, from camera input to lane detection using color filtering and line detection to enable smart lighting autonomous driving in Duckietown.
Figure 1. Image Processing Pipeline for Duckiebot Lane Detection.
PID controller
Figure 2. PID controller.
A diagram illustrating the open loop control system of the Duckietown street lighting system, showing the interaction between light sources and the image processing pipeline.
Figure 3. Open Loop Control of Duckietown Street Lighting System.
A pair of wooden streetlight prototypes designed by Aurel Neff, positioned in Duckietown to provide lighting for smart lighting autonomous driving experiments.
Figure 4. Aurel Neff's Wooden Light Stands for Duckietown.
Open loop of lighting system
Fig.5 Open loop of lighting system
A control loop diagram showing the Duckiebot as the sole sensor for managing street lighting in Duckietown, using detected segments to control lighting conditions.
Figure 6. Control Loop with Duckiebot as the Only Sensor.
A feedback loop diagram showing how the Duckiebot controls its own smart lighting system using detected lane segments and environmental lighting conditions.
Figure 7. Feedback Control Loop of Duckiebot Lighting System.
A control loop diagram showing the watchtower's camera acting as a sensor to manage street lighting in Duckietown, influencing the Duckiebot's lane detection.
Figure 8. Control Loop with Watchtower Camera as Sensor.
A feedback loop diagram illustrating how the watchtower's camera acts as a sensor to control the street lighting system in Duckietown.
Figure 9. Feedback Loop with Watchtower Camera as Sensor.
A control loop diagram showing how the RGB sensor in the watchtower is used to manage street lighting conditions in Duckietown.
Figure 10. Control Loop Using RGB Sensor as Sensor.
A graphical representation showing the effect of updated color ranges on the color detection capabilities of the Duckiebot in Duckietown.
Figure 13. Impact of Modified Color Ranges on Detection.
Investigated lighting conditions in RGB space. The colored dots illustrates the colors of the edges of the grid.
Figure 14. Investigated lighting conditions in RGB space. The colored dots illustrates the colorsof the edges of the grid
A comparison of (a) lane locations considered valid and (b) segments detected by the Duckiebot in Duckietown.
Figure 15. Filtering of Detected Segments in Duckietown.
An image depicting the experimental setup used to assess optimal lighting conditions for the Duckiebot on a straight street in Duckietown.
Figure 16. Experimental Setup for Evaluating Optimal Lighting Conditions on a Straight Street.
An image depicting the experimental setup used to assess the Duckiebot lighting system, highlighting that the WT04 watchtower is not operational.
Figure 17. Experimental Setup for Evaluating the Duckiebot Lighting System with Non-Functional WT04.
Measured values for d and phi while the Duckiebot is following the lane
Figure 18. Lane following performance of the Duckiebot at different and changing light condition of the ceiling light and the street lighting system controlling the light on the streets of Duckietown

Why day and night autonomous driving in Duckietown?

Autonomous driving is already inherently hard. Driving at night makes it even more challenging! This is why smart lighting is an interesting application that intersects with autonomous driving: having city infrastructure, such as traffic lights and watchtowers, generate dynamically varying light – only where and when they’re needed – to make driving at night not only possible but safe. Here are some reasons for which this project is interesting:

Realistic driving scenarios: autonomous driving systems must handle varying lighting conditions. Day and night cycles are just the beginning: transitions like sunrise or sunset make the spectrum of experimental corner cases more complex, hence Duckietown a valuable testbed.

Robust lane-following capabilities: developing an adaptive lighting system in which the city infrastructure “collaborates” with Duckiebot to provide optimal driving scenarios reinforces driving performances and general robustness for lane following.  

Decentralized control for scalability: a decentralized approach to managing lighting implies that the system can be scalable across Duckietowns of arbitrary dimensions, making it more adaptable and resilient.

Autonomous lighting management: a responsive street lighting system, working in tandem with the Duckiebot’s onboard sensors, improves energy efficiency and ensures safety by adjusting to local lighting needs automatically.

Learn about Advanced Autonomy Approaches with Duckietown

Smart Lighting: Realistic Day and Night in Duckietown - the challenges

Implementing smart lighting in Duckietown to improve autonomous driving during day and night cycles presents several challenges. Here are a few examples: 

Hardware modifications: while Duckiebots are equipped with controllable LEDs, city infrastructure does not possess lighting capabilities out of the box. The first step is integrating light sources in the design of Duckietown’s city infrastructure.

Variable lighting conditions: Duckiebots, which in this project rely uniquely on vision in their autonomy pipeline, must adapt to changing lighting conditions such as full darkness, sunrise, sunset, and artificial lighting, which impacts camera vision and lane detection accuracy.

Decentralized control: managing street lighting in a decentralized way across Duckietown ensures that each area adapts to its local lighting needs, compensating for example for the presence of passing Duckiebots with their own lights on. Join control algorithms including both city infrastructure and vehicle lighting intensity add complexity to the system’s design and coordination.

Scalability: the street lighting system must be scalable across the entire city, requiring a design that can be expanded without significant complications.

Safe and reliable operation: the system needs to be safe, adapting to issues such as occasional watchtower lighting source failure, while ensuring consistent lane-following performance.

Smart Lighting: Realistic Day and Night in Duckietown: Results

Smart Lighting: Realistic Day and Night in Duckietown: Authors

David Müller

David Müller is a former Duckietown student of class Autonomous Mobility on Demand at ETH Zurich, and currently works as a Research Engineer at Disney Research, Switzerland.

Learn more

Duckietown is a modular, customizable and state-of-the-art platform for creating and disseminating robotics and AI learning experiences.

It is designed to teach, learn, and do research: from exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge.

Get Started
More Projects
Get Robots