Multi-Agent Reinforcement Learning for Autonomous Driving in Signal-Free Intersections and Roundabouts

Haoran Tang

doi:10.71222/ymabk538

Authors

Haoran Tang University of Birmingham, Birmingham, UK Author

DOI:

https://doi.org/10.71222/ymabk538

Keywords:

signal-free intersections, MARL, LCF

Abstract

Signal-Free Intersections & Roundabouts: One of the most difficult problems that a self-driving car faces is the proper functioning at intersections with no traffic lights. It is because there are no traffic lights, which assist us in making quick decisions, unlike just using lidar sensors and cameras for fixed-lane driving. In these situations the current research on multi-agent reinforcement learning (MARL) does not extent to such domains because of issues like non-stationarity and partial observability. The novel hierarchical learning framework for related communications integrates hierarchical learning structures and complex cost functions, which can better adjust a more adaptive response to safety and traffic flow efficiency.

References

1. R. Chandra, and D. Manocha, "Gameplan: Game-theoretic multi-agent planning with human drivers at intersections, roundabouts, and merging," IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 2676-2683, 2022. doi: 10.1109/lra.2022.3144516

2. M. Mamlouk, and B. Souliman, "Effect of traffic roundabouts on accident rate and severity in Arizona," Journal of Transportation Safety & Security, vol. 11, no. 4, pp. 430-442, 2019. doi: 10.1080/19439962.2018.1452812

3. M. L. Littman, "Markov games as a framework for multi-agent reinforcement learning," In Machine learning proceedings 1994, 1994, pp. 157-163. doi: 10.1016/b978-1-55860-335-6.50027-1

4. L. Busoniu, R. Babuska, and B. De Schutter, "A comprehensive survey of multiagent reinforcement learning," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 38, no. 2, pp. 156-172, 2008. doi: 10.1109/tsmcc.2007.913919

5. A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, "CARLA: An open urban driving simulator," In Conference on robot learning, October, 2017, pp. 1-16.

6. R. Lowe, Y. I. Wu, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch, "Multi-agent actor-critic for mixed cooperative-competitive environments," Advances in neural information processing systems, vol. 30, 2017.

7. J. Chen, S. E. Li, and M. Tomizuka, "Interpretable end-to-end urban autonomous driving with latent deep reinforcement learning," IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 6, pp. 5068-5078, 2021. doi: 10.1109/tits.2020.3046646

8. X. Ma, J. Li, M. J. Kochenderfer, D. Isele, and K. Fujimura, "Reinforcement learning for autonomous driving with latent state inference and spatial-temporal relationships," In 2021 IEEE International Conference on Robotics and Automation (ICRA), May, 2021, pp. 6064-6071. doi: 10.1109/icra48506.2021.9562006

9. J. Wu, Z. Song, and C. Lv, "Deep reinforcement learning-based energy-efficient decision-making for autonomous electric vehicle in dynamic traffic environments," IEEE Transactions on Transportation Electrification, vol. 10, no. 1, pp. 875-887, 2023. doi: 10.1109/tte.2023.3290069