NeoPlanner - Continually Learning Planning Agent for Large Environments guided by LLMs
25Excited to have Swarna present his work on state space graph-based Planning, and how he used an Explore-Exploit approach to build and use this state space graph for future action planning.
Speaker Profile - Swarna Kamal Paul completed his PhD on general problem solving agents from Jadavpur University on 2023. He also have 15 years of work experience in IT industry including consultancy, research and software application development. Currently he is affiliated to TCS India. His research interests includes adaptable agents, LLM based agentic systems, general AI, integrative AI.
Abstract - Sequential planning in large state space and action space quickly becomes intractable due to combinatorial explosion of the search space. To solve this problem I propose a hybrid agent - βneoplannerβ, that synergizes both state space search with queries to foundational LLM to get the best action plan. The reward signals (wherever obtained) from the environment are quantitatively used to drive the search. In places where random exploration is needed, the LLM is queried to generate an action plan. Observations from environment during exploration are converted to learnings and stored in text format as memory. These are eventually used to refine the search. Experiments in the ScienceWorld environment reveals a 124% improvement from the current best method in terms of average reward gained across multiple tasks.
Paper - https://arxiv.org/abs/2312.07368
Code - https://github.com/swarna-kpaul/neoplanner
~~~
My related works referred to in this discussion:
TaskGen: https://www.youtube.com/watch?v=O_XyTT7QGH4
Learning, Fast and Slow: https://www.youtube.com/watch?v=DSVFA7nmwHQ
~~~
0:00 Introduction
1:03 What is Planning
6:05 Why is Planning hard?
17:45 ScienceWorld
20:51 Problems with Prior Methods
28:10 Problems with Planning with LLMs
32:03 Overall Method Overview
34:30 State Space Search
47:26 State Space Search Equations
1:04:05 Learning Value Functions via TD Learning
1:07:21 Overall Agent Architecture
1:15:10 Exploration Prompt
1:22:13 Experiment Details
1:24:27 Discussion
~~~
AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.
Discord: https://discord.gg/bzp87AHJy5
LinkedIn: https://www.linkedin.com/in/chong-min-tan-94652288/
Online AI blog: https://delvingintotech.wordpress.com/
Twitter: https://twitter.com/johntanchongmin
Try out my games here: https://simmer.io/@chongmin