Robust and reliable models are needed to understand how coastlines will evolve over the coming decades, driven by both natural variability and climate change. This study evaluated how accurately five popular ‘reduced-complexity’ models replicate multi-decadal shoreline change at Narrabeen-Collaroy Beach, a sandy embayment in Sydney, Australia. Measured shoreline positions derived from approximately monthly field surveys were used for 20-year calibration and 20-year validation periods. The models performed similarly on average but with large variability between transects. The set-up of several models was modified to compensate for their sensitivity to imperfect input wave data, and further site-specific improvements were identified. Capturing interannual to decadal-scale variability in cross-shore and longshore dynamics at this site was challenging for all five models. Models appeared to aggregate key processes at this timescale into parameter values rather than representing them directly. This suggests time-varying parameters or changes to model structure may be necessary for decadal-scale simulations.