Robust automatic detection of surface and air objects in a maritime environment is a problem that is of growing importance to the Royal Netherlands Navy (RNLN). Due to a shift in the field of operation from the open oceans towards the littoral waters, the RNLN is forced to operate in complex environments with cluttered backgrounds and the presence of many small vessels and a wide range of other objects. Traditional radar systems are not optimal in these circumstances due to their minimum detection range, lack of sensitivity to small, non-metallic, objects and poor classification power. Complementation by Electro-Optical (EO) camera systems is therefore desired, which resulted in the start of the development of a detection algorithm based on polynomial background estimation. Automated object detection in the maritime environment is a complex problem however, due to various complicating factors. These factors include the highly dynamic background, camera motion, the variety in possible objects and their appearance, and the diversity in meteorological as well as environmental circumstances. Although the developed detection algorithm is quite well capable of detecting the objects, it also produces an extensive amount of false detections. This study investigates whether these false detections can be eliminated, while maintaining the true detections, by means of classification of the detections as either target or background. To this end, the initial detection algorithm is optimised to detect as much objects as possible in a carefully constructed dataset of eight hundred Visible Light (VL) images. The resulting detections from the optimised algorithm are used accordingly to train and test various basic classifiers, using a set of features found in the literature. The best performing classifier is selected and the performance of this classifier, and the two-stage detection algorithm as a whole, is subsequently further analysed by means of various tests involving the features used, the evaluation procedure and the fusion of detection results. Results show that especially the features as well as the clustering procedure for detected pixels are important parameters with respect to a good performance of the algorithm. This works shows that the linear discriminant classifier is best to use with the problem among the classifiers considered. Moreover, it is demonstrated that including features of histogram equalized boxes in combination with features of the entire image increased the performance the most, that determining the features on a slightly larger area than the originally detected area is beneficial and that fusion of detections after classification can be used to optimise the detector output. Although the developed classification approach is capable of eliminating many false detections and to retain a majority of the true detections, further research is required. Suggested are separate classifiers for the sea- and sky part, inclusion of the time dimension, optimisation of the operating point of the classifier and preprocessing steps.