In the Western Scheldt estuary, like in many estuaries, safe navigation, flood protection, and ecological targets require a balanced and sustainable sediment management. A thorough understanding of the morphodynamic functioning of the estuary and its response to changes in hydrodynamics (natural sediment transport) and large-scale interventions is imperative. This paper presents a detailed overview of over 65 years of morphological changes and a comprehensive sediment budget of the Western Scheldt estuary that is based on analysis of a unique series of frequent bathymetric surveys and historical data on human–sediment interactions of dredging, dredge disposal and sand mining. Solving the sediment budget reveals an annual sediment import of 2.2 million m3. The highest sediment imports occurred between 1980–1994 and 2005–2020 (2.9 and 3.7 million m3/year). A 1.3 million m3/year net export prevailed between 1994 and 2005. Natural variations in the hydrodynamics (e.g., tidal asymmetry and amplification) and sediment transports cannot explain the derived temporal variations in sediment import rates. Anthropogenic activities play a dominant role. Capital dredging of the main navigation channel has led to channel deepening and significantly increased dredge and disposal volumes. Disposal on tidal flats and in the secondary channel leads to a long-term storage of sand and, consequently, a local increase in bed level and a sand deficit in the remainder of the system that results in increased sediment imports. Large-scale disposal in the western part of the estuary can (temporarily) reverse the sediment exchange between the estuary and its mouth area, as observed between 1994 and 2005. Apparently, large-scale anthropogenic reallocation of sediment by dredging and/or disposal as part of navigation channel improvement, sand mining or nourishment essentially influences the morphological development of the Western Scheldt estuary. This reveals responsibilities as well as opportunities of sediment management for the Western Scheldt and similar estuaries worldwide.