In the last years, the popularity of video-on-demand services has been constantly increasing, especially for the young audiences who are more adept at using new technologies. Through those platforms, the viewers have access to a huge volume of movies at any moment that makes the viewing decision for most of them a very challenging task. Recommender systems are employed by video-on-demand providers to address the former challenge. We propose a novel movie recommender system that filters movies based on the genre-related visual elements of their trailers. The proposed system utilizes a 3D pre-trained deep ConvNet to extract spatio-temporal deep features from the trailers which then are combined, through a Deep Bag of Segments (DBoS) pooling network, with the genre information of the movie to provide a single movie representation. The 3D deep visual genre-aware representation is exploited by a pure content-based filtering system to provide personalized recommendations to users. We conduct offline experiments with two datasets to evaluate the performance of our approach with respect to accuracy and beyond accuracy metrics. We also conduct an online experiment in a real-world streaming platform to evaluate the user perceived utility of the recommendations produced by a pure content-based recommender system using our proposed genre-aware movie descriptor against the same system using genre and visual 3D deep features. We conclude that a continuous genre representation, which reflects genre specific visual elements of the movie, provides interesting results in the content-based movie recommendation task. Exploring further its potential could bring important benefits to various tasks in the movie domain.