As urban environments become increasingly populated and automobile traffic soars, with US citizens spending on average 54 hours a year stuck on the roads, active traffic control management promises to mitigate traffic jams while maintaining (or improving) safety. 

LibSignal++ is a Duckietown-based testbed for reproducible and low-cost sim-to-real evaluation of traffic signal control (TSC) algorithms. Using Duckietown enables consistent, small-scale deployment of both rule-based and learning-based TSC models.

LibSignal++ integrates visual control through camera-based sensing and object detection via the YOLO-v5 model. It features modular components, including Duckiebots, signal controllers, and an indoor positioning system for accurate vehicle trajectory tracking. The testbed supports dynamic scenario replication by enabling both manual and automated manipulation of sensor inputs and road layouts.

Key aspects of the research include:

• Sim-to-real pipeline for Reinforcement Learning (RL)-based traffic signal control training and deployment
• Multi-simulator training support with SUMO, CityFlow, and CARLA
• Reproducibility through standardized and controllable physical components
• Integration of real-world sensors and visual control systems
• Comparative evaluation using rule-based policies on 3-way and 4-way intersections

The work concludes with plans to extend to Machine Learning (ML)-based TSC models and further sim-to-real adaptation.

Training agents for robotics applications requires a substantial amount of data, which is typically costly to collect in the real world. Running simulations is, therefore a logical approach to training agents. But to what degree do simulations provide information that correctly predicts behavior in the real world? In other words, how well do “things” learned in simulation transfer to reality? Sim2Real transfer is an exciting topic and an active area of research.

Simulation-based reinforcement learning often encounters transfer failures due to discrepancies between simulated and real-world dynamics.

This work introduces a method for model adaptation using Latent-State Dynamics Residuals, which correct transition functions in a learned latent space. A latent-variable world model, DRAW, is trained in simulation using variational inference to encode high-dimensional observations into compact multi-categorical latent variables.

The forward dynamics are modeled via autoregressive prediction of latent transitions. A residual learning function is trained on a small, offline real-world dataset without reward supervision to adjust the simulated dynamics. The resulting model, ReDRAW, modifies the forward dynamics logits using residual corrections and enables policy training via actor-critic reinforcement learning on imagined rollouts.

The reward model is reused from the simulation without retraining. To generate diverse training data, the method uses Plan2Explore, which promotes exploration by maximizing model uncertainty. Visual encoders trained in simulation are reused for real-world inputs through zero-shot perception transfer, without fine-tuning.

The approach avoids explicit observation-space correction and operates entirely in the latent space, achieving efficient sim-to-real policy deployment.