Axial Attention & MetNet: A Neural Weather Model for Precipitation Forecasting

Channel:
Switzerland Yannic Kilcher
Subscribers:
291,000
Published on ● Video Link: https://www.youtube.com/watch?v=lmAj0SU_bW0


Duration: 32:34
2,961 views
111

MetNet is a predictive neural network model for weather prediction. It uses axial attention to capture long-range dependencies. Axial attention decomposes attention layers over images into row-attention and column-attention in order to save memory and computation.

https://ai.googleblog.com/2020/03/a-neural-weather-model-for-eight-hour.html
https://arxiv.org/abs/1912.12180

Abstract:
Weather forecasting is a long standing scientific challenge with direct social and economic impact. The task is suitable for deep neural networks due to vast amounts of continuously collected data and a rich spatial and temporal structure that presents long range dependencies. We introduce MetNet, a neural network that forecasts precipitation up to 8 hours into the future at the high spatial resolution of 1 km2 and at the temporal resolution of 2 minutes with a latency in the order of seconds. MetNet takes as input radar and satellite data and forecast lead time and produces a probabilistic precipitation map. The architecture uses axial self-attention to aggregate the global context from a large input patch corresponding to a million square kilometers. We evaluate the performance of MetNet at various precipitation thresholds and find that MetNet outperforms Numerical Weather Prediction at forecasts of up to 7 to 8 hours on the scale of the continental United States.

Authors: Casper Kaae Sønderby, Lasse Espeholt, Jonathan Heek, Mostafa Dehghani, Avital Oliver,Tim Salimans, Shreya Agrawal, Jason Hickey, Nal Kalchbrenner

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher



Other Videos By Yannic Kilcher


2020-04-11CURL: Contrastive Unsupervised Representations for Reinforcement Learning
2020-04-10Enhanced POET: Open-Ended RL through Unbounded Invention of Learning Challenges and their Solutions
2020-04-09Evolving Normalization-Activation Layers
2020-04-08[Drama] Who invented Contrast Sets?
2020-04-07Evaluating NLP Models via Contrast Sets
2020-04-06POET: Endlessly Generating Increasingly Complex and Diverse Learning Environments and Solutions
2020-04-03Dream to Control: Learning Behaviors by Latent Imagination
2020-04-02Can we Contain Covid-19 without Locking-down the Economy?
2020-04-01State-of-Art-Reviewing: A Radical Proposal to Improve Scientific Publication
2020-03-31Agent57: Outperforming the Atari Human Benchmark
2020-03-30Axial Attention & MetNet: A Neural Weather Model for Precipitation Forecasting
2020-03-25[Rant] coronavirus
2020-03-23Online Education - How I Make My Videos
2020-02-24Deep Learning for Symbolic Mathematics
2020-02-21NeurIPS 2020 Changes to Paper Submission Process
2020-02-12Growing Neural Cellular Automata
2020-02-11Turing-NLG, DeepSpeed and the ZeRO optimizer
2020-01-27[Interview] Mark Ledwich - Algorithmic Extremism: Examining YouTube's Rabbit Hole of Radicalization
2020-01-22Reformer: The Efficient Transformer
2020-01-10Go-Explore: a New Approach for Hard-Exploration Problems
2019-12-11NeurIPS 19 Poster Session


Tags:
deep learning
machine learning
arxiv
google
attention mechanism
attention
transformer
rnn
recurrent
weather
long-range
layers
convolutions
cnns
rain
physics