Learn from just Memory Storage and Retrieval: Generative Agents Interacting in Simulation!
20Memory plays a huge role in our lives. What if I told you that just by changing your memory and initial biases, I can change your behavior?
That is just what a group of researchers at Stanford and Google Brain did. They built a sandbox simulator like the Sims, initialized 25 agents with different personalities and relationships, with different goals in life. They let the agents run free and use natural language to select actions, as well as talk to one another if they are in proximity and agree to talk. They let them accumulate information about the world, perform reflections to consolidate memory, and use memory retrieval to select actions (similar to Retrieval-Augmented Generation, except that we are generating actions).
Most importantly, everything is stored in a hierarchy. Memories build upon each other. World representations are symbolic and form part-whole relationships naturally. Action selection and daily planning is also done in a hierarchy from coarse grain to the details.
This work is very interesting and paves the way ahead for memory-based learning. There are still many improvements to be made, like scoring importance of memory as classification instead of rating (yes, I tested it and it works better), implementing a better forgetting mechanism based on Ebbinghaus' forgetting curve, removing redundant memories, having a "conscious" stream to process environmental interactions instead of just extraction from the memory stream, and many more. That said, I still like this work a lot and it is synergistic with a lot of the concepts about memory I want to research on.
2023 will be all about memory, as we unlock the potential of Large Language Models (LLMs) with better grounding through memory.
~~~~~~~~~~~~~~~~~~~~
Paper: https://arxiv.org/abs/2304.03442
Paper Demo (Watch re-run of simulation): https://reverie.herokuapp.com/arXiv_Demo/
Slides: https://github.com/tanchongmin/TensorFlow-Implementations/blob/main/Paper_Reviews/Agent%20Simulation%20Slides.pdf
Harry Potter ChatGPT Text-based RPG game: https://www.youtube.com/watch?v=yF-bXyImLio&list=PLcORGO1bTgjWPj_wup5-XtFGNELuN35ou
A Github repo to recreate this (by mkturkcan, using Alpaca instead of OpenAI ChatGPT): https://github.com/mkturkcan/generative-agents
References:
Socratic Models (various modalities talk to each other in text): https://arxiv.org/abs/2204.00598
GPT4 can do some zero-shot classification: https://www.youtube.com/watch?v=C0Eug9XpcBo
GPT4 can zero-shot some of the Abstraction and Reasoning Corpus (ARC) Challenge: https://www.youtube.com/watch?v=vt2yG1da8Fg
OpenAI Vector embeddings: https://www.youtube.com/watch?v=lIoLCip0HwM
Learning, Fast and Slow (my own hypothesis of how we use memory): https://www.youtube.com/watch?v=Hr9zW7Usb7I
~~~~~~~~~~~~~~~~~~~~~
0:00 Introduction
1:45 Demo of Simulation (Overview)
5:08 Demo of Simulation (State Details and Memory)
14:00 How Agents are Prompted for Actions
20:14 Motivation of the memory-based learning
23:08 Recap on Memory
26:44 Can we learn from just memories alone?
36:10 Is memory related to personality?
39:00 Overall Architecture of Generative Agents
42:30 The power of prompting: Agent character, memories and background
44:42 Inter-Agent Communication
47:09 Taking Control of an Agent
50:11 Initial prompting can lead to cascade of actions
52:28 Memory Retrieval for Planning
1:00:05 Memory Retrieval Limit and Memory Stream
1:03:42 Agent’s prompt to rate importance of memory (and how zero-shot classification is actually better)
1:08:17 Reflections to consolidate memory
1:11:41 Planning for overall grounding of agent’s actions
1:14:01 Trees of Reflection
1:17:28 Symbolic Representation: Perceiving and Acting
1:21:42 Obtaining fine-grained actions via recursive prompting
1:25:22 Is the memory retrieval good?
1:28:15 Are the actions generated plausible?
1:29:24 Personal insight: how to better store and retrieve memories
1:31:20 Discussion
~~~~~~~~~~~~~~~~
AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.
Discord: https://discord.gg/fXCZCPYs
LinkedIn: https://www.linkedin.com/in/chong-min-tan-94652288/
Online AI blog: https://delvingintotech.wordpress.com/
Twitter: https://twitter.com/johntanchongmin
Try out my games here: https://simmer.io/@chongmin