Intelligence = Sampling + Filtering
27Recent work has shown that increased sampling can solve more of the Abstraction and Reasoning Challenge, solve dynamically changing mazes, solve programming problems, solve math olympiad problems etc.
Can just simply increasing sampling give massive performance gains, or is it more than that?
Many thanks to Derek and others for the wonderful discussion!
Related Video (A Roadmap for AI): https://www.youtube.com/watch?v=LuEyoLgxNkc
Slides: https://github.com/tanchongmin/TensorFlow-Implementations/blob/main/Paper_Reviews/Intelligence_Is_Sampling_And_Filtering.pdf
~~~
Paper links:
AlphaCode: https://arxiv.org/abs/2203.07814
Learning, Fast and Slow: https://arxiv.org/abs/2301.13758
gpt-4o for ARC: https://redwoodresearch.substack.com/p/getting-50-sota-on-arc-agi-with-gpt
LLMs as Multiple Expert System to solve ARC: https://arxiv.org/abs/2310.05146
Tree of Thoughts: https://arxiv.org/abs/2305.10601
MCTSr: https://arxiv.org/abs/2406.07394
~~~
0:00 Introduction
1:24 Thought Experiment - Cliff
3:01 Minicolumns and Multiple Sampling
9:33 Thought Experiment - Aiming a Ball
17:19 AlphaCode
37:45 Learning, Fast and Slow
46:28 ARC Challenge
49:50 GPT-4o on ARC: Ryan Greenblatt
55:56 LLMs as a System of Multiple Expert Agents
1:02:27 Segway to Efficient Sampling
1:05:24 Breadth First Search vs Depth First Search
1:10:22 Tree of Thought
1:15:00 Monte Carlo Tree Search
1:21:33 AlphaGo / AlphaZero
1:27:17 MCTSr
1:29:25 My Thoughts
1:33:24 Discussion
2:00:19 Conclusion
~~~
AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.
Discord: https://discord.gg/bzp87AHJy5
LinkedIn: https://www.linkedin.com/in/chong-min-tan-94652288/
Online AI blog: https://delvingintotech.wordpress.com/
Twitter: https://twitter.com/johntanchongmin
Try out my games here: https://simmer.io/@chongmin