Freight mode choice models are traditionally estimated using discrete choice models such as the Multinomial Logit (MNL), valued for their interpretability but often limited in their predictive accuracy and ability to capture complex, nonlinear relationships. Recent studies have applied machine learning to freight mode choice using disaggregate shipment-level data. A gap remains in assessing how machine learning models trained on aggregate data perform and how they compare with MNL models for policy applications. This study addresses this gap by training three machine learning models (logistic regression, Random Forest, and XGBoost) on EU aggregate freight flow data and evaluating them with reference to the NEAC MNL model. The models are assessed using seven criteria identified as relevant for freight policy analysis: predictive performance, interpretability, practicality, computation time, robustness, generalizability, and data efficiency. The results show that XGBoost achieves the highest predictive performance, while logistic regression demonstrates advantages in generalizability, robustness, and data efficiency. The findings highlight the trade-off between predictive performance and interpretability/practicality, indicating that machine learning models can complement but not replace Logit-based models in freight policy analysis.