FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

Channel:
Switzerland Yannic Kilcher
Subscribers:
284,000
Published on ● Video Link: https://www.youtube.com/watch?v=eYgPJ_7BkEw


Duration: 20:12
15,386 views
479

FixMatch is a simple, yet surprisingly effective approach to semi-supervised learning. It combines two previous methods in a clever way and achieves state-of-the-art in regimes with few and very few labeled examples.

Paper: https://arxiv.org/abs/2001.07685
Code: https://github.com/google-research/fixmatch

Abstract:
Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model's performance. In this paper, we demonstrate the power of a simple combination of two common SSL methods: consistency regularization and pseudo-labeling. Our algorithm, FixMatch, first generates pseudo-labels using the model's predictions on weakly-augmented unlabeled images. For a given image, the pseudo-label is only retained if the model produces a high-confidence prediction. The model is then trained to predict the pseudo-label when fed a strongly-augmented version of the same image. Despite its simplicity, we show that FixMatch achieves state-of-the-art performance across a variety of standard semi-supervised learning benchmarks, including 94.93% accuracy on CIFAR-10 with 250 labels and 88.61% accuracy with 40 -- just 4 labels per class. Since FixMatch bears many similarities to existing SSL methods that achieve worse performance, we carry out an extensive ablation study to tease apart the experimental factors that are most important to FixMatch's success. We make our code available at this https URL.

Authors: Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, Colin Raffel

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher



Other Videos By Yannic Kilcher


2020-04-25How much memory does Longformer use?
2020-04-24Supervised Contrastive Learning
2020-04-23Thinking While Moving: Deep Reinforcement Learning with Concurrent Control
2020-04-22[Rant] The Male Only History of Deep Learning
2020-04-21Gradient Surgery for Multi-Task Learning
2020-04-20Longformer: The Long-Document Transformer
2020-04-20Backpropagation and the brain
2020-04-18Shortcut Learning in Deep Neural Networks
2020-04-17Feature Visualization & The OpenAI microscope
2020-04-16Datasets for Data-Driven Reinforcement Learning
2020-04-15FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
2020-04-14Imputer: Sequence Modelling via Imputation and Dynamic Programming
2020-04-13The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
2020-04-12Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discovery
2020-04-11CURL: Contrastive Unsupervised Representations for Reinforcement Learning
2020-04-10Enhanced POET: Open-Ended RL through Unbounded Invention of Learning Challenges and their Solutions
2020-04-09Evolving Normalization-Activation Layers
2020-04-08[Drama] Who invented Contrast Sets?
2020-04-07Evaluating NLP Models via Contrast Sets
2020-04-06POET: Endlessly Generating Increasingly Complex and Diverse Learning Environments and Solutions
2020-04-03Dream to Control: Learning Behaviors by Latent Imagination


Tags:
deep learning
machine learning
arxiv
google
semi-supervised
unlabeled
augmentation
research
randaugment