Learning Phase-Based Descriptions for Action Recognition
Omar Hommos (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Jan van Gemert – Mentor
Silvia Pintea – Mentor
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Action recognition continues to receive significant attention from the research community, with new neural network architectures being developed continuously. Optical flow is by far the most popular input motion representation to these architectures, leaving a lot of undiscovered potential for other types of motion representations. Eulerian representations have recently showed huge improvements in areas like motion magnification and video frame interpolation. This work proposes using a phase-based approach to make the best out of Eulerian motion information. We do this by learning complex filters using complex convolutional layers. Phase descriptions are extracted from the feature maps of these complex layers, and are then passed to the remainder of the convolutional network. Our approach shows great potential, and its performance exceeds that of a single optical flow frame input. We provide detailed analysis on using phase-based methods for Eulerian representations, in addition to further analysis on using Eulerian phase, rather than Lagrangian optical flow, for action recognition.