Title
Video BagNet: Short temporal receptive fields increase robustness in long-term action recognition
Author
Strafforello, O. (TU Delft Pattern Recognition and Bioinformatics; TNO)
Liu, X. (TU Delft Pattern Recognition and Bioinformatics) 
Schutte, Klamer (TNO)
van Gemert, J.C. (TU Delft Pattern Recognition and Bioinformatics)
Contributor
Ceballos, Cristina (editor)
Date
2023
Abstract
Previous work on long-term video action recognition relies on deep 3D-convolutional models that have a large temporal receptive field (RF). We argue that these models are not always the best choice for temporal modeling in videos. A large temporal receptive field allows the model to encode the exact sub-action order of a video, which causes a performance decrease when testing videos have a different sub-action order. In this work, we investigate whether we can improve the model robustness to the sub-action order by shrinking the temporal receptive field of action recognition models. For this, we design Video BagNet, a variant of the 3D ResNet-50 model with the temporal receptive field size limited to 1, 9, 17 or 33 frames. We analyze Video Bag-Net on synthetic and real-world video datasets and experimentally compare models with varying temporal receptive fields. We find that short receptive fields are robust to sub-action order changes, while larger temporal receptive fields are sensitive to the sub-action order.
To reference this document use:
http://resolver.tudelft.nl/uuid:3ae2da81-d8ac-47cc-9ad0-e6b03724c81c
DOI
https://doi.org/10.1109/ICCVW60793.2023.00023
Publisher
IEEE, Piscataway
Embargo date
2024-06-25
ISBN
979-8-3503-0745-0
Source
Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)
Event
2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2023-10-02 → 2023-10-06, Paris, France
Bibliographical note
Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Part of collection
Institutional Repository
Document type
conference paper
Rights
© 2023 O. Strafforello, X. Liu, Klamer Schutte, J.C. van Gemert