Transformers (Part 2)

Channel:
Singapore John Tan Chong Min
Subscribers:
5,390
Published on ● Video Link: https://www.youtube.com/watch?v=oq0vj2pLrHQ


Duration: 1:04:20
128 views
1

Illustrating how a Transformer works from the very basics.

This covers self-attention, multi-headed attention, intuition behind dot product and query, key, value, BERT masked language modelling.

Part 1 here: https://www.youtube.com/watch?v=iBamMr2WEsQ

ChatGPT video here: https://www.youtube.com/watch?v=wA8rjKueB3Q

Slides can be found on:
https://github.com/tanchongmin/TensorFlow-Implementations

0:14 Recap (Word Embedding, Feed Forward Neural Network)
4:00 Intuition behind Query, Key, Value
14:35 Mapping of dimension to Q, K, V
20:50 Intuition behind Dot Product
27:11 Intuition behind scaling of softmax
32:35 Adding up the final values after attention
36:28 Multi-headed attention
43:33 Visualizing 2 attention heads
46:13 Skip-connection and Layer Normalization
50:58 Intuition behind Positional Encoding
54:20 BERT
1:01:40 Final Comments



Other Videos By John Tan Chong Min


2022-06-30Solving the game of Nim with Monte Carlo / Monte Carlo Tree Search (Part 2)
2022-06-30Solving the game of Nim with Monte Carlo / Monte Carlo Tree Search (Part 2)
2022-06-30Solving the game of Nim with Monte Carlo / Monte Carlo Tree Search (Part 2)
2022-06-30Solving the game of Nim with Monte Carlo / Monte Carlo Tree Search
2022-06-27Solving the Game of Nim with Value Estimates
2022-06-27Solving the Game of Nim with Monte Carlo / Monte Carlo Tree Search (Part 1)
2022-06-27Solving the Game of Nim with Monte Carlo / Monte Carlo Tree Search (Part 2)
2022-06-22Learning via Self-Play: An AlphaGo/AlphaZero Story
2022-06-08Geometric Deep Learning (Part 1)
2022-06-08Geometric Deep Learning (Part 2)
2022-06-06Transformers (Part 2)
2022-06-01Equation Solver!
2022-05-30Transformers (Part 1)
2022-05-30Can a Neural Network model if-else statements?
2022-05-29Minesweeper AI using Neural Network!
2022-05-29Minesweeper AI using Neural Network! (Part 2)
2022-05-28CodinGame : Let's try to solve the Minesweeper Puzzle! (Part 2)
2022-05-28CodinGame: Let's try to solve the Minesweeper Puzzle!
2022-05-15CodinGame: Pushing rank (again) in Clash of Code!
2022-05-09Recurrent Neural Networks (RNN)
2022-05-01CodinGame: Spring Challenge - Gold League Attempt!