Satellite remote sensing is becoming a widely used monitoring technique in coastal sciences. Yet, no benchmarking studies exist that compare the performance of popular satellite-derived shoreline mapping algorithms against standardized sets of inputs and validation data. Here we present a new benchmarking framework to evaluate the accuracy of shoreline change observations extracted from publicly available satellite imagery (Landsat and Sentinel-2). Accuracy and precision of five established shoreline mapping algorithms are evaluated at four sandy beaches with varying geologic and oceanographic conditions. Comparisons against long-term in situ beach surveys reveal that all algorithms provide horizontal accuracy on the order of 10 m at microtidal sites. However, accuracy deteriorates as the tidal range increases, to more than 20 m for a high-energy macrotidal beach (Truc Vert, France) with complex foreshore morphology. The goal of this open-source, collaborative benchmarking framework is to identify areas of improvement for present algorithms, while providing a stepping stone for testing future developments, and ensuring reproducibility of methods across various research groups and applications.