Transformers Encoders: How To Make Your Own (And Why You Should)

Channel:
IMJONEZZ
Subscribers:
825
Published on ● Video Link: https://www.youtube.com/watch?v=mm21JVh-rAQ


Category:
Guide
Duration: 52:13
83 views
4

Build your own Masked Language Model using BERT (RoBERTA)

Learn about training a custom tokenizer and making sure it works with your model. Create custom functions for tokenizing your chosen text (for us it's the entire works of Emmanuel Kant). Fine-tune and transfer learn your model for a downstream task afterwards!

One thing this includes that I haven't seen anywhere else is a guide on how to actually load your models after saving them, that used to be a lot harder and it bothers me when I see tutorials that don't include that.



Other Videos By IMJONEZZ


2022-10-26Stable Diffusion in 5 Levels of Difficulty: Level 2
2022-10-25Stable Diffusion in 5 Levels of Difficulty: Level 1
2022-10-14Sometimes this game sucks my soul
2022-10-11Assassin's Creed Valhalla: What happens when you finish the Tombs of the Fallen?
2022-10-10How to make an RNN (LSTM) in Python (while True: learn() part 6 final)
2022-10-07How to make a model that uses Stochastic Gradient Descent in Python (while True: learn() part 5)
2022-10-07How to make a Genetic Algorithm System in Python (while True: learn() part 4)
2022-10-05Big Boy Legos
2022-10-01Transformers Encoder and Decoders: How To Make Your Own (And Why You Should)
2022-09-30Transformers Decoders: How To Make Your Own (And Why You Should)
2022-09-23Transformers Encoders: How To Make Your Own (And Why You Should)
2022-05-10Machine Learning in one line of code
2022-05-04Lego Star Wars: The Skywalker Saga - All Cutscenes (Episode 1)
2022-05-04Lego Star Wars: The Skywalker Saga - All Current Codes
2021-09-15First go
2021-09-12Skyrim Multiplayer: Jarl Balgruuuuuuuuuf the sitting #2
2021-09-09Skyrim Multiplayer: Adventure Begins #1
2021-07-13What do you get for Ymir's Tears short
2021-07-13Far Cry Just Vibing
2021-07-12Dying Light is a Perfectly Balanced game
2021-06-17Assassin's Creed Valhalla: How to get GOLD on Bear Mastery Challenge Templebrough Fort


Tags:
encoder and decoder
machine learning
neural network
transformer
encoder only
decoder and encoder