Action recognition continues to receive significant attention from the research community, with new neural network architectures being developed continuously. Optical flow is by far the most popular input motion representation to these architectures, leaving a lot of undiscovered potential for other types of motion representations. Eulerian representations have recently showed huge improvements in areas like motion magnification and video frame interpolation. This work proposes using a phase-based approach to make the best out of Eulerian motion information. We do this by learning complex filters using complex convolutional layers. Phase descriptions are extracted from the feature maps of these complex layers, and are then passed to the remainder of the convolutional network. Our approach shows great potential, and its performance exceeds that of a single optical flow frame input. We provide detailed analysis on using phase-based methods for Eulerian representations, in addition to further analysis on using Eulerian phase, rather than Lagrangian optical flow, for action recognition.