FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding

Channel:
Canada LLMs Explained - Aggregate Intellect - AI.SCIENCE
Subscribers:
22,300
Published on ● Video Link: https://www.youtube.com/watch?v=3McC_PjeRYY


Duration: 1:00:35
606 views
13

For slides and more information on the paper, visit https://ai.science/e/fine-gym-fine-gym-a-dataset-for-fine-grained-video-action-understanding-and-our-experience-of-building-a-high-quality-dataset--rS08FM6kLXXd0MLjTBpm

Discussion lead: Dian Shao (PhD Candidate, CUHK)
Discussion moderator: Xiyang Chen (CTO, Aggregate Intellect)

We will be hosting another livestream session featuring Dian Shao from CUHK, speaking about her team's latest work FineGym, a fine-graned action understanding dataset that received 3 "strongly accept" scores at CVPR this year. Dian will also be sharing their experience and lessons learned from building a high quality dataset.

Join us live here: https://ai.science/e/fine-gym-fine-gym-a-dataset-for-fine-grained-video-action-understanding-and-our-experience-of-building-a-high-quality-dataset--rS08FM6kLXXd0MLjTBpm
Link to the paper's homepage: https://sdolivia.github.io/FineGym/
What will be discussed?

- Introduction to FineGym
- Why it is important to go fine-grained for action understanding tasks
- Lessons learned from creating a high quality dataset
- How to strike a balance between accuracy and efficiency on subtly different actions?
- How to model complex temporal dynamics efficiently, effectively and robustly
- Future work of action understanding

Abstract

On public benchmarks, current action recognition techniques have achieved great success. However, when used in real-world applications, e.g. sport analysis, which requires the capability of parsing an activity into phases and differentiating between subtly different actions, their performances remain far from being satisfactory. To take action recognition to a new level, we develop FineGym, a new dataset built on top of gymnastic videos. Compared to existing action recognition datasets, FineGym is distinguished in richness, quality, and diversity. In particular, it provides temporal annotations at both action and sub-action levels with a three-level semantic hierarchy. For example, a "balance beam" event will be annotated as a sequence of elementary sub-actions derived from five sets: "leap-jump-hop", "beam-turns", "flight-salto", "flight-handspring", and "dismount", where the sub-action in each set will be further annotated with finely defined class labels. This new level of granularity presents significant challenges for action recognition, e.g. how to parse the temporal structures from a coherent action, and how to distinguish between subtly different action classes. We systematically investigate representative methods on this dataset and obtain a number of interesting findings. We hope this dataset could advance research towards action understanding.



Other Videos By LLMs Explained - Aggregate Intellect - AI.SCIENCE


2020-05-27Model Selection for Optimal Prediction in Statistical Learning - Part 2 / 2 | AISC
2020-05-26Representation Learning of Histopathology Images using Graph Neural Networks | AISC
2020-05-26BillionX acceleration using AI Emulators | AISC
2020-05-22Machine Learning Methods for High Throughput Virtual Screening with a focus on Organic Photovoltaics
2020-05-21Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer
2020-05-20Reinforcement Learning for Batch-to-Batch Bioprocess Optimisation | AISC
2020-05-20Leaf Doctor: Plant Disease Detection Using Image Classification | Deep Learning Workshop Capstone
2020-05-20News ScanNER: Entity Tagging in News Headlines | Deep Learning Workshop Capstone
2020-05-19New methods for identifying latent manifold structure from neural data | ASIC
2020-05-19Using unsupervised machine learning to uncover hidden scientific knowledge | AISC
2020-05-15FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding
2020-05-14A Literature Review on ML in Climate Science | AISC
2020-05-13[cnvrg.io] Operating System for Machine Learning | AISC
2020-05-12Tobias Pfaff (DeepMind): Learning to Simulate Complex Physics with Graph Networks
2020-05-07Multi Type Mean Field Reinforcement Learning | AISC
2020-05-07Proving the Lottery Ticket Hypothesis: Pruning is All You Need | AISC Livestream with the Author
2020-05-06Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthe
2020-05-06A Literature Review on Deep Learning in Finance | AISC
2020-05-02COVID19 and AI: Ethics and Data Rights Panel | AISC & NYAI
2020-04-30A Literature Review on Interpretability for Machine Learning | AISC
2020-04-29A Literature Review on ML in Health Care : Introducing new AISC Stream | AISC


Tags:
machine learning
deep learning
video action understanding
computer vision