High-level planning with large language models - SayCan
16How can we plan using a language model? SayCan offers us a way to do long-term planning using a language model grounded with affordances to determine feasible actions. The language model provides the flexibility of interpreting user input by natural language, while the affordance function helps to ensure that only feasible actions are chosen.
This is a form of hierarchical planning, whereby the actions are tasks to fulfil for the lower-level systems. Here, the lower-level systems are modular skills and are trained with reinforcement learning/pre-defined. In order to map these lower-level systems to higher level planning, we utilize SayCan to select the right actions to fulfil the overall goal.
On a related note, the idea of grounding language models in affordances might also help constrain the outputs of language models such as ChatGPT to make it more reliable.
~~~~
Relevant Reading:
SayCan: https://say-can.github.io/
SayCan Presentation Slides: https://github.com/tanchongmin/TensorFlow-Implementations/blob/main/Paper_Reviews/SayCan%20Slides.pdf
Learning, Fast and Slow (my idea of using two systems for learning, of which SayCan can help with making arbitrary goals quantifiable): https://www.youtube.com/watch?v=Hr9zW7Usb7I
Inner Monologue (using self-thought to improve performance): https://arxiv.org/abs/2207.05608
Socratic Model (multiple domain experts coming to a decision): https://arxiv.org/abs/2204.00598
Chain of Thought Reasoning: https://arxiv.org/abs/2201.11903
PaLM language model: https://arxiv.org/abs/2204.02311
Pick and Place architecture:
ViLD (to detect objects): https://arxiv.org/abs/2104.13921
CLIPort (action from text): https://cliport.github.io/
Transporter Networks (spatial symmetries for policies): https://transporternets.github.io/
~~~~
0:00 Introduction
2:19 Main Question
7:55 Preliminary Video
9:17 LLMs may not output valid actions
16:02 Language and Affordance
26:14 Training Value Function with Reinforcement Learning
32:50 SayCan Overview
35:48 Evaluating SayCan on Example Tasks
50:38 SayCan Overall Algorithm
1:02:43 Code Walkthrough
1:18:50 Kitchen Environment
1:22:53 Results
1:33:04 Chain-of-thought reasoning
1:49:36 Connection to Fast & Slow
1:55:45 Connection to Reliable GPT
1:58:08 Discussion
2:22:50 Conclusion
~~~~
AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.
Discord: https://discord.gg/fXCZCPYs
LinkedIn: https://www.linkedin.com/in/chong-min-tan-94652288/
Online AI blog: https://delvingintotech.wordpress.com/.
Twitter: https://twitter.com/johntanchongmin
Try out my games here: https://simmer.io/@chongmin