Set Constrained Temporal Transformer for Set Supervised Action Segmentation | AISC
3Speaker(s): Mohsen Fayyaz
Facilitator(s):
Find the recording, slides, and more info at https://ai.science/e/set-constrained-temporal-transformer-for-set-supervised-action-segmentation--u3zQtOEisdNINJaLEnKU
Motivation / Abstract
Temporal action segmentation is a topic of increasing interest, however, annotating each frame in a video is cumbersome and costly. Weakly supervised approaches therefore aim at learning temporal action segmentation from videos that are only weakly labeled. In this work, we assume that
for each training video only the list of actions is given that occur in the video, but not when, how often, and in which order they occur. In order to address this task, we propose an approach that can be trained end-to-end on such data. The approach divides the video into smaller temporal regions
and predicts for each region the action label and its length. In addition, the network estimates the action labels for each frame. By measuring how consistent the frame-wise predictions are with respect to the temporal regions and the annotated action labels, the network learns to divide a video
into class-consistent regions. We evaluate our approach on three datasets where the approach achieves state-of-the-art results.
What was discussed?
- Temporal Action Segmentation
- Weakly-supervised approaches
- Variable-length video processing for action recognition
- Video Understanding Problems
------
#AISC hosts 3-5 live sessions like this on various AI research, engineering, and product topics every week! Visit https://ai.science for more details