Hippocampal Replay for Learning (Full Length with Questions)

Channel:
Singapore John Tan Chong Min
Subscribers:
6,300
Published on ● Video Link: https://www.youtube.com/watch?v=SG02XgfzxEg


Duration: 53:33
525 views
8

Using Hippocampal Replay to Consolidate Experiences in Memory-Augmented Reinforcement Learning (Paper ID 38)

See updated ideas here in RL Fast and Slow: https://www.youtube.com/watch?v=M10f3ihj3cE
3 min video summary: https://www.youtube.com/watch?v=lm5ozEzoolE
Paper link: https://openreview.net/forum?id=RAOVIJ8rZR
Go-Explore Explanation: https://www.youtube.com/watch?v=oyyOa_nJeDs
Code: https://github.com/tanchongmin/Hippocampal-Replay
Slides: https://github.com/tanchongmin/TensorFlow-Implementations/tree/main/Paper_Reviews

#MemARI_2022

Brief description:
Traditional Reinforcement Learning (RL) agents have difficulty learning from a sparse reward signal. To overcome this, we use a similar memory augmentation mechanism as Go-Explore, and store the most competent trajectories in memory. In order to enable consistent performance, we use hippocampal replay (preplay to consolidate states, replay to update memory of states) to generate an "exploration highway" to facilitate exploration of good states in the future. Such a method of performing hippocampal replay leads to consistent performance (higher solve rate), and less exploration (higher minimum number of steps to solve).

0:00 Introduction
2:00 Go-Explore (Recap)
9:45 Agents Used
10:55 Selection Function
15:25 Environments Used
16:45 Memory Initialization and Updates
19:00 Hippocampal Replay
25:59 Exploration Highway
30:44 Results
36:08 Hyperparameter Tuning Effects
40:38 Goal-Directed Intrinsic Reward
50:38 Discussion

~~~~~~~~~~~~~~~~~~

AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.

Discord: https://discord.gg/fXCZCPYs
LinkedIn: https://www.linkedin.com/in/chong-min-tan-94652288/
Online AI blog: https://delvingintotech.wordpress.com/.
Twitter: https://twitter.com/johntanchongmin
Try out my games here: https://simmer.io/@chongmin



Other Videos By John Tan Chong Min


2023-02-21High-level planning with large language models - SayCan
2023-02-13Learning, Fast and Slow: Towards Fast and Adaptable Agents in Changing Environments
2023-02-07Using Logic Gates as Neurons - Deep Differentiable Logic Gate Networks!
2023-01-31Learn from External Memory, not just Weights: Large-Scale Retrieval for Reinforcement Learning
2023-01-17How ChatGPT works - From Transformers to Reinforcement Learning with Human Feedback (RLHF)
2023-01-09HyperTree Proof Search - Automated Theorem Proving with AlphaZero and Transformers!
2022-12-23CodinGame Fall Challenge 2022: A First Look (managed to get to Silver!)
2022-12-21Can ChatGPT solve CodinGame/Google Kickstart problems?
2022-12-19Reinforcement Learning Fast and Slow: Goal-Directed and Memory Retrieval Mechanism!
2022-12-12A New Framework of Memory for Learning (Part 1)
2022-11-14Hippocampal Replay for Learning (Full Length with Questions)
2022-11-14Hippocampal Replay for Learning (3 min summary)
2022-11-07AlphaTensor: Using Reinforcement Learning for Efficient Matrix Multiplication
2022-10-27Playing Go on TyGem and learning from AI (~ 3 kyu)
2022-10-13Heroes of Might and Magic III - Armageddon's Blade Campaign (First Playthrough) - Final!!!
2022-10-13Heroes of Might and Magic III - Armageddon's Blade Campaign (First Playthrough) - Part 6
2022-10-11Playing Go on Tygem + AI Analysis (~4 kyu)
2022-10-11Heroes of Might and Magic III - Armageddon's Blade Campaign (First Playthrough) - Part 5
2022-10-11Heroes of Might and Magic III - Armageddon's Blade Campaign (First Playthrough) - Part 4
2022-10-10Playing Go on Tygem + AI Analysis (~4 kyu)
2022-10-10Heroes of Might and Magic III - Armageddon's Blade Campaign (First Playthrough) - Part 3