Are current long-term video understanding datasets long-term?

Conference Paper (2023)
Author(s)

O. Strafforello (TNO, TU Delft - Pattern Recognition and Bioinformatics)

Klamer Schutte (TNO)

J.C. Gemert (TU Delft - Pattern Recognition and Bioinformatics)

Research Group
Pattern Recognition and Bioinformatics
Copyright
© 2023 O. Strafforello, Klamer Schutte, J.C. van Gemert
DOI related publication
https://doi.org/10.1109/ICCVW60793.2023.00319
More Info
expand_more
Publication Year
2023
Language
English
Copyright
© 2023 O. Strafforello, Klamer Schutte, J.C. van Gemert
Research Group
Pattern Recognition and Bioinformatics
Bibliographical Note
Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.@en
Pages (from-to)
2959-2968
ISBN (print)
979-8-3503-0745-0
ISBN (electronic)
979-8-3503-0744-3
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Many real-world applications, from sport analysis to surveillance, benefit from automatic long-term action recognition. In the current deep learning paradigm for automatic action recognition, it is imperative that models are trained and tested on datasets and tasks that evaluate if such models actually learn and reason over long-term information. In this work, we propose a method to evaluate how suitable a video dataset is to evaluate models for long-term action recognition. To this end, we define a long-term action as excluding all the videos that can be correctly recognized using solely short-term information. We test this definition on existing long-term classification tasks on three popular real-world datasets, namely Breakfast, CrossTask and LVU, to determine if these datasets are truly evaluating long-term recognition. Our study reveals that these datasets can be effectively solved using shortcuts based on short-term information. Following this finding, we encourage long-term action recognition researchers to make use of datasets that need long-term information to be solved.

Files

Are_current_long-term_video_un... (pdf)
(pdf | 2.45 Mb)
- Embargo expired in 25-06-2024
License info not available