The authors note with alarm that statistical noise caused by statistical incompetence is beginning to creep into research on cost overrun in public investment projects, contaminating research with work that does not meet basic standards of validity and reliability. The paper gives examples of such work and proposes three heuristics to root out the problem. First, researchers who are not statisticians, or do not have a strong background in statistics, should abstain from doing statistical analysis, and instead rely on more experienced colleagues, preferably professional statisticians. Second, journal referees should clearly state their level of statistical proficiency to journal editors, so these can set the right referee team. Finally, journal editors should make sure that at least one referee is capable of reviewing the statistical and methodological aspects of a paper. The work under review would have benefitted from observing these simple heuristics, as would any work based on statistical analysis.@en

This paper gives an overview of good and bad practice for understanding and curbing cost overrun in large capital investment projects, with a critique of Love and Ahiaga-Dagbui (2018) as point of departure. Good practice entails: (a) Consistent definition and measurement of overrun; in contrast to mixing inconsistent baselines, price levels, etc. (b) Data collection that includes all valid and reliable data; as opposed to including idiosyncratically sampled data, data with removed outliers, non-valid data from consultancies, etc. (c) Recognition that cost overrun is systemically fat-tailed; in contrast to understanding overrun in terms of error and randomness. (d) Acknowledgment that the root cause of cost overrun is behavioral bias; in contrast to explanations in terms of scope changes, complexity, etc. (e) De-biasing cost estimates with reference class forecasting or similar methods based in behavioral science; as opposed to conventional methods of estimation, with their century-long track record of inaccuracy and systemic bias. Bad practice is characterized by violating at least one of these five points. Love and Ahiaga-Dagbui violate all five. In so doing, they produce an exceptionally useful and comprehensive catalog of the many pitfalls that exist, and must be avoided, for properly understanding and curbing cost overrun.

@en