General Information

Title: Semantic Image Segmentation Methods in the Duckietown Project
Authors: Kristina S. Lanchukovskaya; Dasha E. Shabalina; Tatiana V. Liakh
Institution: Novosibirsk State University, The Russian Federation
Citation: Lanchukovskaya, K.S., Shabalina, D.E. and Liakh, T.V., 2022, June. Semantic image segmentation methods in the duckietown project. In 2022 IEEE 23rd International Conference of Young Professionals in Electron Devices and Materials (EDM) (pp. 611-617). IEEE.

Semantic Image Segmentation Methods in Duckietown

In Duckietown, where self-driving agents (i.e., Duckiebots) operate in structured environments, segmentation is essential for lane detection, object recognition, and obstacle avoidance. Semantic Image Segmentation assigns a class label to each pixel in an image, allowing autonomous systems to interpret their surroundings.

This research evaluates four deep learning models – SegNet, U-Net, FC-DenseNet, and DeepLab-v3 by comparing their efficiency, accuracy, and real-time applicability. Understanding the trade-offs between these models helps optimize perception for Duckiebots navigating the Duckietown.

These models rely on Convolutional Neural Networks (CNNs) to extract hierarchical features. SegNet prioritizes memory efficiency, U-Net incorporates skip connections for improved localization, FC-DenseNet enhances feature reuse through dense connectivity, and DeepLab-v3 captures multi-scale context with atrous spatial pyramid pooling. Each model presents a balance between computational cost and segmentation accuracy, influencing its suitability for embedded systems like Duckiebots.

Implementing semantic segmentation in Duckietown enhances autonomy by enabling self-driving agents to interpret complex visual inputs. The selection of an appropriate segmentation model depends on processing constraints and real-time performance needs. By integrating optimized segmentation techniques, Duckiebots improve decision-making in structured environments.

Highlights - Semantic Image Segmentation Methods in Duckietown

Here is a visual tour of the work of the authors. For all the details, check out the full paper.

Abstract

In the author’s words:

The article focuses on evaluation of the applicability of existing semantic segmentation algorithms for the Duckietown simulator. Duckietown is an open research project in the field of autonomously controlled robots. The article explores classical semantic image segmentation algorithms. Their analysis for applicability in Duckietown is carried out.

With the help of them, we want to make a dataset for training neural networks. The following was investigated: edge-detection techniques, threshold algorithms, region growing, segmentation algorithms based on clustering, neural networks. The article also reviewed networks designed for semantic image segmentation and machine learning frameworks, taking into account all the limitations of the Duckietown simulator.

Experiments were conducted to evaluate the accuracy of semantic segmentation algorithms on such classes of Duckietown objects as road and background. Based on the results of the analysis, region growing algorithms and clustering algorithms were selected and implemented.

Experiments were conducted to evaluate the accuracy on such classes of Duckietown objects as road, background and traffic signs. After evaluating the accuracy of the algorithms considered, it was decided to use Color segmentation, Mean Shift, Thresholding algorithms and Segmentation of signs by April-tag for image preprocessing. For neural networks, experiments were conducted to evaluate the accuracy of semantic segmentation algorithms on such classes of Duckietown objects as road and background. After evaluating the accuracy of the algorithms considered, it was decided to select the DeepLab-v3 neural network. Separate module was created for semantic image segmentation in Duckietown.

Conclusion - Semantic Image Segmentation Methods in Duckietown

Here are the conclusions from the author of this paper:

The article analyzes the applicability of semantic segmentation algorithms in the Duckietown simulator, which simulates autopilot robots in an urban environment.

It was found that methods based on classical computer vision algorithms are inferior to methods based on neural networks in terms of stability, segmentation accuracy and speed of operation. It was proposed to use classical computer vision algorithms for marking images and preparing datasets and neural networks for segmentation on robots.

CV algorithms taking into account the features of the Duckietown simulator. Thus, classical computer vision algorithms, such as area-building algorithms and clustering algorithms, were chosen for image preprocessing. OpenCV and Scikit-image libraries were selected for the experiment. The best result during the testing was obtained using MeanShift and cv2.threshold together, and road signs were segmented most successfully using April tag.

Also, after testing the selected neural networks, it was decided to select the DeepLab-v3 neural network as an adapted semantic segmentation algorithm for the Duckietown simulator. After testing the trained DeepLab-v3 neural network model on Duckiebot, a separate module for semantic image segmentation was created in the Duckietown open research project. In the future, it is planned to add such classes of Duckietown objects as a duck in the role of a pedestrian, road markings (red, yellow, white) and Duckiebot.

Project Authors

Kristina S. Lanchukovskaya is affiliated with the department of IT, Novosibirsk State University, Novosibirsk, Russia.

Dasha E. Shabalina is affiliated with the department of IT, Novosibirsk State University, Novosibirsk, Russia.

Tatiana V. Liakh is a Senior Lecturer at the Department of Computer Science, Electrical and Space Engineering, Novosibirsk State University, Novosibirsk, Russia.

Learn more

Duckietown is a platform for creating and disseminating robotics and AI learning experiences.

It is modular, customizable and state-of-the-art, and designed to teach, learn, and do research. From exploring the fundamentals of computer science and automation to pushing the boundaries of knowledge, Duckietown evolves with the skills of the user.