Proving the Lottery Ticket Hypothesis: Pruning is All You Need | AISC Livestream with the Author

Channel:
Canada LLMs Explained - Aggregate Intellect - AI.SCIENCE
Subscribers:
20,500
Published on ● Video Link: https://www.youtube.com/watch?v=JbyEleBP6BI


Duration: 46:31
1,001 views
21

For slides and more information on the paper, visit https://ai.science/e/2020-05-07

Speaker: Eran Malach
Moderator: Nour Fahmy

Abstract

The lottery ticket hypothesis (Frankle and Carbin, 2018), states that a randomly-initialized network contains a small subnetwork such that, when trained in isolation, can compete with the performance of the original network. We prove an even stronger hypothesis (as was also conjectured in Ramanujan et al., 2019), showing that for every bounded distribution and every target network with bounded weights, a sufficiently over-parameterized neural network with random weights contains a subnetwork with roughly the same accuracy as the target network, without any further training.



Other Videos By LLMs Explained - Aggregate Intellect - AI.SCIENCE


2020-05-20Reinforcement Learning for Batch-to-Batch Bioprocess Optimisation | AISC
2020-05-20Leaf Doctor: Plant Disease Detection Using Image Classification | Deep Learning Workshop Capstone
2020-05-20News ScanNER: Entity Tagging in News Headlines | Deep Learning Workshop Capstone
2020-05-19New methods for identifying latent manifold structure from neural data | ASIC
2020-05-19Using unsupervised machine learning to uncover hidden scientific knowledge | AISC
2020-05-15FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding
2020-05-14A Literature Review on ML in Climate Science | AISC
2020-05-13[cnvrg.io] Operating System for Machine Learning | AISC
2020-05-12Tobias Pfaff (DeepMind): Learning to Simulate Complex Physics with Graph Networks
2020-05-07Multi Type Mean Field Reinforcement Learning | AISC
2020-05-07Proving the Lottery Ticket Hypothesis: Pruning is All You Need | AISC Livestream with the Author
2020-05-06Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthe
2020-05-06A Literature Review on Deep Learning in Finance | AISC
2020-05-02COVID19 and AI: Ethics and Data Rights Panel | AISC & NYAI
2020-04-30A Literature Review on Interpretability for Machine Learning | AISC
2020-04-29A Literature Review on ML in Health Care : Introducing new AISC Stream | AISC
2020-04-29A Literature Review on ML in Neuroscience - Introducing new AISC Stream | AISC
2020-04-28Model Selection for Optimal Prediction in Statistical Learning (Part 1: 7 Wheels of Stat Learning)
2020-04-28A Literature Review on Machine Learning in Materials Science | AISC
2020-04-23SELFIES: A 100% robust representation of semantically constrained Graphs, for deep generative models
2020-04-23A Literature Review on Reinforcement Learning in Process Control | AISC