Can LLMs be used in self-driving? CoMAL: Collaborative Multi-Agent LLM for Mixed Autonomy Traffic
7CoMAL is a very interesting paper which uses mutli-agent collaboration to define leader / follower roles between autonomous vehicles, and then each agent will plan their velocity, acceleration and spacing from car in front individually.
It has a memory to draw information from and to update dynamically according to environment experience.
Overall, there is a lot of merit to the architecture created - although I would say there needs to be more rule-based components, especially when numerical figures are involved as LLMs are not great with numbers.
The future will be a close-knit integration between symbolic rule-based approaches and LLM as the cognitive layer above, with multiple memory abstraction spaces to draw upon.
Slides: https://github.com/tanchongmin/agentjo/blob/main/paper_reviews/CoMAL.pdf
Paper: https://arxiv.org/abs/2410.14368
GitHub: https://github.com/Hyan-Yao/CoMAL
~~~
0:00 Introduction
1:45 Environments
4:47 Key Points: RL vs LLMs
11:34 Overall Framework of CoMAL
23:42 Multi-Agent Collaboration
26:37 Single Agent Pipeline
28:31 System Prompt
31:11 Collaboration between Agents
34:04 Reasoning to Planning
42:45 Memory
47:00 Benchmarks and Results
1:08:45 Discussion
1:50:35 Closing Remarks
~~~
AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.
Discord: https://discord.gg/bzp87AHJy5
LinkedIn: https://www.linkedin.com/in/chong-min-tan-94652288/
Online AI blog: https://delvingintotech.wordpress.com/
Twitter: https://twitter.com/johntanchongmin
Try out my games here: https://simmer.io/@chongmin