GPT-NeoX-20B - Open-Source huge language model by EleutherAI (Interview w/ co-founder Connor Leahy)

Channel:
Switzerland Yannic Kilcher
Subscribers:
284,000
Published on ● Video Link: https://www.youtube.com/watch?v=AJwnbSP_rq8


Duration: 20:06
18,839 views
668

#eleuther #gptneo #gptj

EleutherAI announces GPT-NeoX-20B, a 20 billion parameter open-source language model, inspired by GPT-3. Connor joins me to discuss the process of training, how the group got their hands on the necessary hardware, what the new model can do, and how anyone can try it out!

OUTLINE:
0:00 - Intro
1:00 - Start of interview
2:00 - How did you get all the hardware?
3:50 - What's the scale of this model?
6:00 - A look into the experimental results
11:15 - Why are there GPT-Neo, GPT-J, and GPT-NeoX?
14:15 - How difficult is training these big models?
17:00 - Try out the model on GooseAI
19:00 - Final thoughts

Read the announcement: https://blog.eleuther.ai/announcing-20b/
Try out the model: https://goose.ai/
Check out EleutherAI: https://www.eleuther.ai/
Read the code: https://github.com/EleutherAI/gpt-neox
Hardware sponsor: https://www.coreweave.com/

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n



Other Videos By Yannic Kilcher


2022-02-21Listening to You! - Channel Update (Author Interviews)
2022-02-20All about AI Accelerators: GPU, TPU, Dataflow, Near-Memory, Optical, Neuromorphic & more (w/ Author)
2022-02-18[ML News] Uber: Deep Learning for ETA | MuZero Video Compression | Block-NeRF | EfficientNet-X
2022-02-17CM3: A Causal Masked Multimodal Model of the Internet (Paper Explained w/ Author Interview)
2022-02-16AI against Censorship: Genetic Algorithms, The Geneva Project, ML in Security, and more!
2022-02-15HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learning (w/ Author)
2022-02-10[ML News] DeepMind AlphaCode | OpenAI math prover | Meta battles harmful content with AI
2022-02-08Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents (+Author)
2022-02-07OpenAI Embeddings (and Controversy?!)
2022-02-06Unsupervised Brain Models - How does Deep Learning inform Neuroscience? (w/ Patrick Mineault)
2022-02-04GPT-NeoX-20B - Open-Source huge language model by EleutherAI (Interview w/ co-founder Connor Leahy)
2022-01-29Predicting the rules behind - Deep Symbolic Regression for Recurrent Sequences (w/ author interview)
2022-01-27IT ARRIVED! YouTube sent me a package. (also: Limited Time Merch Deal)
2022-01-25[ML News] ConvNeXt: Convolutions return | China regulates algorithms | Saliency cropping examined
2022-01-21Dynamic Inference with Neural Interpreters (w/ author interview)
2022-01-19Noether Networks: Meta-Learning Useful Conserved Quantities (w/ the authors)
2022-01-11This Team won the Minecraft RL BASALT Challenge! (Paper Explanation & Interview with the authors)
2022-01-05Full Self-Driving is HARD! Analyzing Elon Musk re: Tesla Autopilot on Lex Fridman's Podcast
2022-01-02Player of Games: All the games, one algorithm! (w/ author Martin Schmid)
2021-12-30ML News Live! (Dec 30, 2021) Anonymous user RIPS Tensorflw | AI prosecutors rising | Penny Challenge
2021-12-28GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models


Tags:
deep learning
machine learning
arxiv
explained
neural networks
ai
artificial intelligence
paper
leahy
eleuther
eleutherai
eleuther ai
connor leahy
coreweave
gooseai
goose ai
gpt neo
gpt-neo
gpt-neox
gpt-neox-20b
gpt-j
open source
huggingface
transformer
transformer models
gpt-3
open source gpt-3
download gpt-neox
gpu cluster
large language model
large language models
machine learning tutorial