Predicting the parameters of a neural network without training it | Trustworthy AI | Webinar
0Reducing barriers for deep learning practitioners to develop neural networks is one step towards democratizing deep learning, by making the technology more accessible to smaller players in the field. This AI for Good Discovery will explore the key themes in Graham Taylor’s research around removing barriers for deep learning practitioners who lack the background or resources to work with cutting-edge models that require advanced forms of hardware parallelism. Collaborating with Facebook AI Research (now Meta), his team developed a technique to initialize diverse neural network architectures using a “meta-model”. This research challenges the long-held assumption that gradient-based optimizers are required to train deep neural networks.
Astonishingly, the meta-model can predict parameters for almost any neural network in just one forward pass, achieving ~60% accuracy on the popular CIFAR-10 dataset without any training. Moreover, while the meta-model was training, it did not observe any network close to the ResNet-50 whose ~25 million parameters it predicted. Like the team’s 2020 work to reduce the computational requirements of GANs, this talk highlights the approach which democratizes deep learning by making the technology accessible to smaller players in the field, such as startup companies and not-for-profits. The work appeared at NeurIPS 2021 and was reported on by Anil Ananthaswamy for Quanta Magazine.
🎙 Speaker:
Graham Taylor, Canada Research Chair and Professor of Engineering, University of Guelph
🎙 Moderator:
Wojciech Samek, Head of Department of Artificial Intelligence, @FraunhoferHHI
Join the Neural Network!
👉 https://aiforgood.itu.int/neural-network/
The AI for Good networking community platform powered by AI.
Designed to help users build connections with innovators and experts, link innovative ideas with social impact opportunities, and bring the community together to advance the SDGs using AI.
🔴 Watch the latest #AIforGood videos!
Explore more #AIforGood content:
1️ ⃣ AI for Good Top Hits
• Top Hits
2️ ⃣ AI for Good Webinars
• AI for Good Webinars
3️ ⃣ AI for Good Keynotes
• AI for Good Keynotes
📩 Stay updated and join our weekly AI for Good newslettehttp://eepurl.com/gI2kJ5kJ5
📅 Discover what's next on our programmhttps://aiforgood.itu.int/programme/me/
🗞 Check out the latest AI for Good newhttps://aiforgood.itu.int/newsroom/om/
📱 Explore the AI for Good blohttps://aiforgood.itu.int/ai-for-good-blog/...
🌎 Connect on our social media:
Websithttps://aiforgood.itu.int/nt/
Twittehttps://twitter.com/ITU_AIForGoodd
LinkedIn Paghttps://www.linkedin.com/company/265119077
LinkedIn Grouhttps://www.linkedin.com/groups/85677488
Instagrahttps://www.instagram.com/aiforgoodd
Faceboohttps://www.facebook.com/AIforGoodd
What is AI for Good?
The AI for Good series is the leading action-oriented, global & inclusive United Nations platform on AI. The Summit is organized all year, always online, in Geneva by the ITU with XPRIZE Foundation in partnership with over 35 sister United Nations agencies, Switzerland and ACM. The goal is to identify practical applications of AI and scale those solutions for global impact.
Disclaimer:
The views and opinions expressed are those of the panelists and do not reflect the official policy of the ITU.
#TrustworthyAI #AIforGoodWebinars