Large, long-term coastal imagery datasets are nowadays a low-cost source of information for various coastal research disciplines. However, the applicability of many existing algorithms for coastal image analysis is limited for these large datasets due to a lack of automation and robustness. Therefore manual quality control and site- and time-dependent calibration are often required. In this paper we present a fully automated algorithm that classifies each pixel in an image given a pre-defined set of classes. The necessary robustness is obtained by distinguishing one class of pixels from another based on more than a thousand pixel features and relations between neighboring pixels rather than a handful of color intensities. Using a manually annotated dataset of 192 coastal images, a SSVM is trained and tested to distinguish between the classes water, sand, vegetation, sky and object. The resulting model correctly classifies 93.0% of all pixels in a previously unseen image. Two case studies are used to show how the algorithm extracts beach widths and water lines from a coastal camera station. Both the annotated dataset and the software developed to perform the model training and prediction are provided as free and open-source data sources.