Failure of residential buildings during floods is an important cause of damage and loss of life. In the case of the Netherlands, the collapse of buildings is implicitly included in current damage and mortality curves since these are generated from historical data. However, the Netherlands has not experienced destructive flooding since 1953, so damage functions for modern buildings do not exist. Therefore, this paper assesses the effect of floods on modern Dutch residences with laboratory tests and structural models in order to formulate physically-based fragility curves. The results gathered are also applicable to similarly-built masonry and cavity-wall rowhouses elsewhere. Almost half of the Dutch population live in terraced houses (also known as townhouses or rowhouses), of which the critical failure mechanism during a flood is out-of-plane bending of the load-bearing walls. Failure of these structural elements should be analysed with the pressure coefficient, Cp, instead of the currently used drag coefficient, CD, because wall collapse is more likely than displacement of the entire structure. This paper describes the quantification of both coefficients by conducting flume experiments on rectangular boxes with different geometries and orientations. Higher drag coefficients are derived from the experiments than provided by FEMA, resulting in higher hydrodynamic loads on the residences. The physical approach to evaluate the collapse of residences is exemplified with a case study of the three most common type of residences in the Netherlands. Structural analyses of their load-bearing walls subjected to a hydrostatic and hydrodynamic load perpendicular to the wall show failure due to milder flood conditions than the current damage curves do for all case study residences. A sensitivity analysis shows an important influence of wall thickness, initial axial loading of the wall, and the flood water level inside the residence.