LLM as Pattern Machines(Part 2) - Goal Directed Decision Transformers, 10-Year Plan for Intelligence
10(Part 1) This is an interesting paper as it shows that LLMs can represent, manipulate, and extrapolate more abstract, nonlinguistic patterns. This is an interesting finding as all along, we have been thinking that LLMs are just great text-based completers for text with some semantic meaning.
However, I show that the methods used in this paper may not be ideal. Firstly, using random tokens is not a good strategy, as the semantic priors of these tokens still get used and may influence the results!
(Part 2) Moreover, using a reward-based approach like in Decision Transformers still takes a long time to learn. I propose a Goal-Directed Decision Transformer instead and show that it outperforms the method used in this paper!
I also provide a 10-year vision of how I think intelligent machines can be created, using concepts that I have learned over various papers and my own experiments.
Using an LLM as a way to associate patterns will likely be the underpinning of intelligence. However, my view is that this approach of using abstract tokens is probably not the right one. We will still need to imbue semantic meaning of some sort for better performance.
~~~~~~~~~~~~~~~~~~~~~~
Paper: https://arxiv.org/abs/2307.04721
Slides: https://github.com/tanchongmin/TensorFlow-Implementations/blob/main/Paper_Reviews/LLMs%20as%20General%20Pattern%20Machines%20(Slides).pdf
My related videos:
LLMs as General Pattern Machines (Part 1): https://www.youtube.com/watch?v=qEoiLgdQC9k
Learning Fast and Slow: https://www.youtube.com/watch?v=Hr9zW7Usb7I
LLMs as a System to solve ARC: https://www.youtube.com/watch?v=plVRxP8hQHY
Decision Transformers: https://www.youtube.com/watch?v=AW7vHggnAps
Voyager (High level LLM-based planning in MineCraft to functional Code): https://www.youtube.com/watch?v=Y-pgbjTlYgk
Ghost in the MineCraft (High level LLM-based planning in MineCraft to list of functions to call): https://www.youtube.com/watch?v=_VXOczXIkks
HyperTree Proof Search: https://www.youtube.com/watch?v=CIGl2NboS9s
GATO (Generalist Agent by DeepMind): https://www.youtube.com/watch?v=ENspggRUs4U
Knowledge Graphs and LLMs: https://www.youtube.com/watch?v=1RZ5yIyz31c
Joint-Embedding Predictive Architecture (Yann LeCun): https://www.youtube.com/watch?v=M98OLk30dBk
Hierarchy and its use in AI: https://www.youtube.com/watch?v=1x049Dmxes0
~~~~~~~~~~~~~~~~~~~~~~~
0:00 Introduction and Recap
3:40 Decision Transformers
7:15 Is Learning Reward Necessary?
14:40 Can LLMs do logic?
29:45 Alternate Rewards for Learning (Goals)
32:10 Experiments of Reward vs Goal Conditioning (Goal Conditioning is Better)
43:08 Semantic-description based prompting improves outcomes!
47:20 How to incorporate goals into Cart Pole
53:43 My 10-Year Vision for Intelligence
1:13:38 Goal-Directed Decision Transformers (looking for collaborators)
1:15:11 Conclusion and Discussion
~~~~~~~~~~~~~~~~~~~~~~~
AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.
Discord: https://discord.gg/bzp87AHJy5
LinkedIn: https://www.linkedin.com/in/chong-min-tan-94652288/
Online AI blog: https://delvingintotech.wordpress.com/
Twitter: https://twitter.com/johntanchongmin
Try out my games here: https://simmer.io/@chongmin