Residential self-selection (RSS) is the theoretical mechanism that explains that the impact of the built environment on travel behaviour is weaker than bivariate correlations suggest, because mode attitudes influence both the built environment and travel behaviour and therefore at least partially account for the bivariate relationship. Recently, the concept of travel-related reasons for residential choice has been introduced, which reflects the actual extent to which the travel-related characteristics of the built environment were considered during the relocation decision. In this paper, we hypothesize that travel-related location reasons are stronger predictors of the built environment choice than generic mode attitudes. This hypothesis is examined by estimating both a cross-sectional and a longitudinal Structural Equation Model using data gathered in the Netherlands. The results suggest that the travel-related location reasons are indeed stronger predictors for built environment location than travel mode attitudes and that the directions of causality between attitudes, travel-related location reasons, the built environment, and travel behaviour often run in both directions. Substantively, our findings indicate that public transport use is most strongly affected by the built environment (after controlling for both stated reasons and attitudes), while car and bicycle use are hardly affected. From a practical point of view, this suggests that transforming the built environment to be more friendly to public transport may increase the use of public transport, but that, at least in the Netherlands, such a strategy would not work well if the aim were to reduce car use or increase bicycle use.