Autolab evaluations underway
We have started evaluating the submissions in our Duckietown “Robotarium” (aka Autolab):
Duckiebot onboard camera feed
Robotarium watchtower camera feed
To queue your submissions for robotarium evaluation, please follow these instructions:
You need to use the –challenge option to specify 3 challenges: the two simulated ones (testing and validation) and the hardware one:
- dts challenges submit –challenge aido2-LF-sim-validation,aido2-LF-sim-testing,aido2-LF-real-validation
- dts challenges submit –challenge aido2-LFV-sim-validation,aido2-LFV-sim-testing,aido2-LFV-real-validation
- dts challenges submit –challenge aido2-LFV-sim-validation,aido2-LFVI-sim-testing,aido2-LFVI-real-validation
We will evaluate submissions by participants that are in the top part of the leaderboard in the simulated testing challenge.
The robotarium evaluations are limited, and we will do them in a round robin strategy for each user. We aim to evaluate all in the top 10 of the simulated challenge; and then more if there is the possibility.
Participants can have multiple submissions in the “real” challenges. We will evaluate first according to “user priority” or by most recent. The priority is settable through the web interface by using the top right button.