Datasets for Data-Driven Reinforcement Learning

Channel:
Switzerland Yannic Kilcher
Subscribers:
284,000
Published on ● Video Link: https://www.youtube.com/watch?v=-h1KB8ps11A


Duration: 19:55
4,504 views
140

Offline Reinforcement Learning has come more and more into focus recently in domains where classic on-policy RL algorithms are infeasible to train, such as safety-critical tasks or learning from expert demonstrations. This paper presents an extensive benchmark for evaluating offline RL algorithms in a variety of settings.

Paper: https://arxiv.org/abs/2004.07219
Code: https://github.com/rail-berkeley/offline_rl

Abstract:
The offline reinforcement learning (RL) problem, also referred to as batch RL, refers to the setting where a policy must be learned from a dataset of previously collected data, without additional online data collection. In supervised learning, large datasets and complex deep neural networks have fueled impressive progress, but in contrast, conventional RL algorithms must collect large amounts of on-policy data and have had little success leveraging previously collected datasets. As a result, existing RL benchmarks are not well-suited for the offline setting, making progress in this area difficult to measure. To design a benchmark tailored to offline RL, we start by outlining key properties of datasets relevant to applications of offline RL. Based on these properties, we design a set of benchmark tasks and datasets that evaluate offline RL algorithms under these conditions. Examples of such properties include: datasets generated via hand-designed controllers and human demonstrators, multi-objective datasets, where an agent can perform different tasks in the same environment, and datasets consisting of a heterogeneous mix of high-quality and low-quality trajectories. By designing the benchmark tasks and datasets to reflect properties of real-world offline RL problems, our benchmark will focus research effort on methods that drive substantial improvements not just on simulated benchmarks, but ultimately on the kinds of real-world problems where offline RL will have the largest impact.

Authors: Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, Sergey Levine

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher



Other Videos By Yannic Kilcher


2020-04-26[Drama] Schmidhuber: Critique of Honda Prize for Dr. Hinton
2020-04-25How much memory does Longformer use?
2020-04-24Supervised Contrastive Learning
2020-04-23Thinking While Moving: Deep Reinforcement Learning with Concurrent Control
2020-04-22[Rant] The Male Only History of Deep Learning
2020-04-21Gradient Surgery for Multi-Task Learning
2020-04-20Longformer: The Long-Document Transformer
2020-04-20Backpropagation and the brain
2020-04-18Shortcut Learning in Deep Neural Networks
2020-04-17Feature Visualization & The OpenAI microscope
2020-04-16Datasets for Data-Driven Reinforcement Learning
2020-04-15FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
2020-04-14Imputer: Sequence Modelling via Imputation and Dynamic Programming
2020-04-13The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
2020-04-12Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discovery
2020-04-11CURL: Contrastive Unsupervised Representations for Reinforcement Learning
2020-04-10Enhanced POET: Open-Ended RL through Unbounded Invention of Learning Challenges and their Solutions
2020-04-09Evolving Normalization-Activation Layers
2020-04-08[Drama] Who invented Contrast Sets?
2020-04-07Evaluating NLP Models via Contrast Sets
2020-04-06POET: Endlessly Generating Increasingly Complex and Diverse Learning Environments and Solutions


Tags:
deep learning
machine learning
reinforcement learning
deep rl
off-policy
on-policy
replay buffer
dataset
benchmark
berkeley
rail
offline
online