Autonomous Lane Following in Duckietown: Our DAgger Approach
Projects ·Ever wondered how miniature autonomous cars learn to navigate? At Duckietown, a platform designed for education and research in robotics, we tackled the challenge of lane following and vehicle avoidance using an advanced imitation learning technique. This project, which was also a baseline for the AI-DO challenge, demonstrates how deep neural networks can learn to drive in both simulated and real-world environments.
The Duckietown Challenge: Driving with AI 🦆
Duckietown features small, self-driving robots (Duckiebots) that operate on a track, needing to stay within lanes and react to other moving vehicles. This platform is ideal for developing and testing autonomous driving algorithms in a simplified yet realistic setting. The core task is to enable the Duckiebot to follow the lane expertly and avoid collisions.
Our Approach: Imitation Learning with DAgger
Our method leveraged Imitation Learning, where an AI agent learns by observing and mimicking an expert’s behavior. In this case, our “expert” was a pure pursuit controller, a classic robotics algorithm known for its precise path tracking capabilities.
However, simple imitation learning can struggle with covariate shift – when the agent encounters states it didn’t see during expert demonstrations, leading to compounding errors. To overcome this, we employed the Dataset Aggregation (DAgger) algorithm (Ross et al., 2011). DAgger is an iterative training process where:
- A model is initially trained on expert demonstrations.
- The model then drives, and if it encounters a situation where it’s unsure, the expert is queried for the correct action.
- This new, challenging data is added to the training dataset, and the model is re-trained.
This cycle helps the model progressively learn from its own mistakes and handle a wider range of scenarios, making it more robust. The code skeleton for our implementation was based on the work of Diaz Cabrera, Manfred Ramon (2018), detailed in their Masters thesis “Interactive and Uncertainty-aware Imitation Learning: Theory and Applications.” (Diaz Cabrera, 2018)
Seeing it in Action: Simulation to Reality 🌐
Our results, showcased in the video below, demonstrate the effectiveness of our DAgger-based approach across different scenarios:
- Simulation Lane Following [00:03]: The video first highlights the Duckiebot autonomously following lanes within the Duckietown simulator. This phase allows for rapid iteration and testing.
- Simulation Lane Following with Vehicles [01:12]: We then ramp up the complexity by introducing other vehicles, requiring the Duckiebot to not only follow its lane but also avoid dynamic obstacles.
- Real Lane Following [01:37]: Finally, the model is deployed on a physical Duckiebot on a real track, demonstrating its ability to generalize from the simulated environment to the physical world, which is often a significant hurdle in robotics.
Watch our Duckiebot navigate the track:
Conclusion
Our work on the Duckietown gym demonstrated that Imitation Learning combined with DAgger is a powerful strategy for developing robust autonomous navigation systems. By interactively collecting data and refining the model’s understanding, we successfully trained a deep neural network capable of lane following and obstacle avoidance in both simulated and real-world Duckietown environments.
For a deeper dive into the technical details and to explore the codebase, you can find the full source code on the Duckietown Challenges GitHub repository.
References
- Ross, S., Gordon, G., & Bagnell, D. (2011). A reduction of imitation learning and structured prediction to no-regret online learning. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, 627–635.
- Diaz Cabrera, M. R. (2018). Interactive and Uncertainty-aware Imitation Learning: Theory and Applications [PhD thesis]. Concordia University.