Voyager - An LLM-based curriculum generator, actor and critic, with skill reuse in Minecraft!

Channel:
Singapore John Tan Chong Min
Subscribers:
5,360
Published on ● Video Link: https://www.youtube.com/watch?v=Y-pgbjTlYgk


The Voyager
Game:
The Voyager (2023)
Duration: 1:26:46
438 views
16

AutoGPT and Baby AGI have all been the hype recently as it extends the capabilities of a single LLM by integrating it in a systems approach, and breaking down a complex goal into more manageable subgoals. This coupled with other improvements in LLM generative abilities such as Chain of Thought reasoning and grounding by observations in environment (ReAct), along with reflecting on the current gameplay (RefleXion), has led to a huge leap in performance. Retrieval-augmented generation can also help condition generation better with more in-context examples.

Here, Voyager incorporates all these, along with an automatic curriculum generator to adjust to the agent's strengths, as well as code-based actions (yes, that is right, an action is defined as a function in Java code), which enable flexible behaviour and reuse. The learning signal is amazingly human-free learning signal as the code generation has grounding in environment feedback and code interpreter feedback, as well as a GPT-4 critique, which tells the agent what to do if the objective is not met.

Though the prompts are still heavily hand-crafted towards MineCraft, the general framework of incorporating LLM for open-world exploration is interesting and can be used to build better and more performant systems in the future!

~~~~~~~~~~~~~~~~~

Slides: https://github.com/tanchongmin/TensorFlow-Implementations/blob/main/Paper_Reviews/Voyager%20GITM%20MineCraft%20Slides.pdf
Part 2 on Ghost In The Minecraft: https://www.youtube.com/watch?v=_VXOczXIkks

Voyager: https://voyager.minedojo.org/
Voyager Code: https://github.com/MineDojo/Voyager
MineFlayer API: https://github.com/PrismarineJS/mineflayer/blob/master/docs/api.md

ReAct: https://react-lm.github.io/
RefleXion: https://github.com/noahshinn024/reflexion
AutoGPT: https://github.com/Significant-Gravitas/Auto-GPT
AgentGPT: https://agentgpt.reworkd.ai/
SayCan: https://github.com/google-research/google-research/blob/master/saycan/SayCan-Robot-Pick-Place.ipynb
SayCan (my video): https://www.youtube.com/watch?v=iS3ikfSsp6Y
Dreamer v3: https://arxiv.org/abs/2301.04104

~~~~~~~~~~~~~~~~~

0:00 Introduction to Voyager, AgentGPT
8:38 Introduction to LLMOps
13:46 My thoughts on Voyager
15:32 Voyager performs well in comparison with LLM-based methods
18:55 Components of Voyager
41:51 What is a skill?
53:15 Automatic Curriculum Generator Details
56:13 Iterative Prompting Details
1:12:58 Background Check: Does GPT4 already have the model answers?
1:14:25 Skill Generation Details
1:18:41 How is Voyager different from other LLM-based methods?
1:21:50 Results
1:24:18 Conclusion

~~~~~~~~~~~~~~~~~~

AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.

Discord: https://discord.gg/bzp87AHJy5
LinkedIn: https://www.linkedin.com/in/chong-min-tan-94652288/
Online AI blog: https://delvingintotech.wordpress.com/
Twitter: https://twitter.com/johntanchongmin
Try out my games here: https://simmer.io/@chongmin



Other Videos By John Tan Chong Min


2023-07-27How Llama 2 works: Ghost Attention, Quality Supervised Fine-tuning, RLHF for Safety and Helpfulness
2023-07-27Llama 2 vs ChatGPT
2023-07-11I-JEPA: Importance of Predicting in Latent Space
2023-07-09Gen AI Study Group Introductory Tutorial - Transformers, ChatGPT, Prompt Engineering, Projects
2023-07-03Tutorial #5: Strict JSON LLM Framework - Get LLM to output JSON exactly the way you want it!
2023-07-01Tutorial #4: SymbolicAI ChatBot In-Depth Demonstration (Tool Use and Iterative Processing)
2023-06-29How do we learn so fast? Towards a biologically plausible model for one-shot learning.
2023-06-20LLMs as a system to solve the Abstraction and Reasoning Corpus (ARC) Challenge!
2023-06-16Tutorial #3: Symbolic AI - Symbols, Operations, Expressions, LLM-based functions!
2023-06-13No more RL needed! LLMs for high-level planning: Voyager + Ghost In the Minecraft
2023-06-06Voyager - An LLM-based curriculum generator, actor and critic, with skill reuse in Minecraft!
2023-06-01Evolution ChatGPT Prompt Game - From Bacteria to.... Jellyfish???
2023-05-30Prompt Engineering and LLMOps: Tips and Tricks
2023-05-25Hierarchy! The future of AI: How it helps representations and why it is important.
2023-05-18Prediction builds representations! Fixed Bias speeds up learning!
2023-05-09Memory: How is it encoded, retrieved and how it can be used for learning systems
2023-05-02I created a Law Court Simulator with GPT4!
2023-05-02I created a Law Court Simulator with ChatGPT!
2023-04-25Creating a ChatGPT Harry Potter Text-based RPG game!
2023-04-25Learn from just Memory Storage and Retrieval: Generative Agents Interacting in Simulation!
2023-04-18The future is neuro-symbolic: Expressiveness of ChatGPT and generalizability of symbols (SymbolicAI)


Other Statistics

The Voyager Statistics For John Tan Chong Min

There are 710 views in 2 videos for The Voyager. About 3 hours worth of The Voyager videos were uploaded to his channel, roughly 1.04% of the content that John Tan Chong Min has uploaded to YouTube.