CURL: Contrastive Unsupervised Representations for Reinforcement Learning

Channel:
Switzerland Yannic Kilcher
Subscribers:
284,000
Published on ● Video Link: https://www.youtube.com/watch?v=hg2Q_O5b9w4


Duration: 28:45
11,007 views
361

Contrastive Learning has been an established method in NLP and Image classification. The authors show that with relatively minor adjustments, CL can be used to augment and improve RL dramatically.

Paper: https://arxiv.org/abs/2004.04136
Code: https://github.com/MishaLaskin/curl

Abstract:
We present CURL: Contrastive Unsupervised Representations for Reinforcement Learning. CURL extracts high-level features from raw pixels using contrastive learning and performs off-policy control on top of the extracted features. CURL outperforms prior pixel-based methods, both model-based and model-free, on complex tasks in the DeepMind Control Suite and Atari Games showing 2.8x and 1.6x performance gains respectively at the 100K interaction steps benchmark. On the DeepMind Control Suite, CURL is the first image-based algorithm to nearly match the sample-efficiency and performance of methods that use state-based features.

Authors: Aravind Srinivas, Michael Laskin, Pieter Abbeel

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher



Other Videos By Yannic Kilcher


2020-04-21Gradient Surgery for Multi-Task Learning
2020-04-20Longformer: The Long-Document Transformer
2020-04-20Backpropagation and the brain
2020-04-18Shortcut Learning in Deep Neural Networks
2020-04-17Feature Visualization & The OpenAI microscope
2020-04-16Datasets for Data-Driven Reinforcement Learning
2020-04-15FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
2020-04-14Imputer: Sequence Modelling via Imputation and Dynamic Programming
2020-04-13The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
2020-04-12Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discovery
2020-04-11CURL: Contrastive Unsupervised Representations for Reinforcement Learning
2020-04-10Enhanced POET: Open-Ended RL through Unbounded Invention of Learning Challenges and their Solutions
2020-04-09Evolving Normalization-Activation Layers
2020-04-08[Drama] Who invented Contrast Sets?
2020-04-07Evaluating NLP Models via Contrast Sets
2020-04-06POET: Endlessly Generating Increasingly Complex and Diverse Learning Environments and Solutions
2020-04-03Dream to Control: Learning Behaviors by Latent Imagination
2020-04-02Can we Contain Covid-19 without Locking-down the Economy?
2020-04-01State-of-Art-Reviewing: A Radical Proposal to Improve Scientific Publication
2020-03-31Agent57: Outperforming the Atari Human Benchmark
2020-03-30Axial Attention & MetNet: A Neural Weather Model for Precipitation Forecasting


Tags:
deep learning
machine learning
rl
reinforcement learning
unsupervised
contrast
contrastive
encoder
self-supervised
deep rl
representation
representation learning
query
key