Reinforcement Learning with Unsupervised Auxiliary Tasks
84https://arxiv.org/abs/1611.05397
Abstract:
Deep reinforcement learning agents have achieved state-of-the-art results by directly maximising cumulative reward. However, environments contain a much wider variety of possible training signals. In this paper, we introduce an agent that also maximises many other pseudo-reward functions simultaneously by reinforcement learning. All of these tasks share a common representation that, like unsupervised learning, continues to develop in the absence of extrinsic rewards. We also introduce a novel mechanism for focusing this representation upon extrinsic rewards, so that learning can rapidly adapt to the most relevant aspects of the actual task. Our agent significantly outperforms the previous state-of-the-art on Atari, averaging 880\% expert human performance, and a challenging suite of first-person, three-dimensional \emph{Labyrinth} tasks leading to a mean speedup in learning of 10× and averaging 87\% expert human performance on Labyrinth.
Authors:
Max Jaderberg, Volodymyr Mnih, Wojciech Marian Czarnecki, Tom Schaul, Joel Z Leibo, David Silver, Koray Kavukcuoglu
Other Videos By Yannic Kilcher
| 2019-02-18 | GPT-2: Language Models are Unsupervised Multitask Learners |
| 2019-02-02 | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift |
| 2019-01-30 | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding |
| 2019-01-09 | What’s in a name? The need to nip NIPS |
| 2018-12-21 | Stochastic RNNs without Teacher-Forcing |
| 2018-12-18 | Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations |
| 2018-04-07 | World Models |
| 2018-03-18 | Curiosity-driven Exploration by Self-supervised Prediction |
| 2017-12-13 | git for research basics: fundamentals, commits, branches, merging |
| 2017-11-28 | Attention Is All You Need |
| 2017-08-28 | Reinforcement Learning with Unsupervised Auxiliary Tasks |
| 2017-08-09 | Learning model-based planning from scratch |
| 2017-08-04 | Imagination-Augmented Agents for Deep Reinforcement Learning |