Transformers (Part 1)

Channel:
Singapore John Tan Chong Min
Subscribers:
5,330
Published on ● Video Link: https://www.youtube.com/watch?v=iBamMr2WEsQ


Duration: 1:01:49
278 views
2

Illustrating how a Transformer works from the very basics.

Covers intuition behind attention, parallel processing rather than hidden state, embedding space, and encoder structure of a Transformer.

Part 2 here: https://www.youtube.com/watch?v=oq0vj2pLrHQ

ChatGPT video here: https://www.youtube.com/watch?v=wA8rjKueB3Q

Slides can be found on:
https://github.com/tanchongmin/TensorFlow-Implementations

0:00 Introduction
1:43 Find Waldo
4:36 Memory Activity
11:25 Problems with RNNs / MLPs for sequential data
14:58 Transformer as a solution to these problems
18:30 AI Dungeon demonstration
27:55 Word tokenisation
33:18 One-hot encoding
34:18 Embedding space
38:12 Overall Transformer Architecture
41:48 Encoder-Decoder Structure
43:30 Self-Attention Block and how it helps with polysemy
1:00:17 Feed-forward block



Other Videos By John Tan Chong Min


2022-06-30Solving the game of Nim with Monte Carlo / Monte Carlo Tree Search (Part 2)
2022-06-30Solving the game of Nim with Monte Carlo / Monte Carlo Tree Search
2022-06-27Solving the Game of Nim with Value Estimates
2022-06-27Solving the Game of Nim with Monte Carlo / Monte Carlo Tree Search (Part 1)
2022-06-27Solving the Game of Nim with Monte Carlo / Monte Carlo Tree Search (Part 2)
2022-06-22Learning via Self-Play: An AlphaGo/AlphaZero Story
2022-06-08Geometric Deep Learning (Part 1)
2022-06-08Geometric Deep Learning (Part 2)
2022-06-06Transformers (Part 2)
2022-06-01Equation Solver!
2022-05-30Transformers (Part 1)
2022-05-30Can a Neural Network model if-else statements?
2022-05-29Minesweeper AI using Neural Network!
2022-05-29Minesweeper AI using Neural Network! (Part 2)
2022-05-28CodinGame : Let's try to solve the Minesweeper Puzzle! (Part 2)
2022-05-28CodinGame: Let's try to solve the Minesweeper Puzzle!
2022-05-15CodinGame: Pushing rank (again) in Clash of Code!
2022-05-09Recurrent Neural Networks (RNN)
2022-05-01CodinGame: Spring Challenge - Gold League Attempt!
2022-05-01CodinGame: Spring Challenge - Can I make it to Gold?
2022-04-27CodinGame: Spring Challenge - Monte Carlo Wild Mana Collection