How much memory does Longformer use?

Channel:
Switzerland Yannic Kilcher
Subscribers:
296,000
Published on ● Video Link: https://www.youtube.com/watch?v=gJR28onlqzs


Duration: 9:18
3,840 views
161

A calculation of the memory requirements of the Longformer.

Original video: https://youtu.be/_8KNb5iqblE
Paper: https://arxiv.org/abs/2004.05150

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher



Other Videos By Yannic Kilcher


2020-05-05TAPAS: Weakly Supervised Table Parsing via Pre-training (Paper Explained)
2020-05-04Chip Placement with Deep Reinforcement Learning (Paper Explained)
2020-05-03I talk to the new Facebook Blender Chatbot
2020-05-02Jukebox: A Generative Model for Music (Paper Explained)
2020-05-01[ML Coding Tips] Separate Computation & Plotting using locals
2020-04-30The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies (Paper Explained)
2020-04-29Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask (Paper Explained)
2020-04-28[Rant] Online Conferences
2020-04-27Do ImageNet Classifiers Generalize to ImageNet? (Paper Explained)
2020-04-26[Drama] Schmidhuber: Critique of Honda Prize for Dr. Hinton
2020-04-25How much memory does Longformer use?
2020-04-24Supervised Contrastive Learning
2020-04-23Thinking While Moving: Deep Reinforcement Learning with Concurrent Control
2020-04-22[Rant] The Male Only History of Deep Learning
2020-04-21Gradient Surgery for Multi-Task Learning
2020-04-20Longformer: The Long-Document Transformer
2020-04-20Backpropagation and the brain
2020-04-18Shortcut Learning in Deep Neural Networks
2020-04-17Feature Visualization & The OpenAI microscope
2020-04-16Datasets for Data-Driven Reinforcement Learning
2020-04-15FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence


Tags:
deep learning
machine learning
nlp
natural language processing
machine translation
arxiv
google
attention mechanism
attention
transformer
tensor2tensor
rnn
recurrent
seq2seq