Advancing reproducibility and replicability in simulation
Challenges and opportunities
Yilin Huang (TU Delft - Technology, Policy and Management)
Deniz Cetinkaya (TU Delft - Technology, Policy and Management, Bournemouth University)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Computer simulation has become increasingly complex and widely applied across different domains. However, the reproducibility and replicability (R&R) of simulation models remain limited. Despite recent improvements, independent reproduction or replication of simulation experiments is rare in the literature. This paper provides an overview of the state of research on R&R in simulation, highlights recent developments, and discusses key concepts and future perspectives. It first examines how R&R has been viewed, approached, and evaluated and then outlines typical challenges and defining characteristics of R&R. Emerging opportunities are also discussed in light of community-driven practices, artificial intelligence, and quantum computing. Given the significant role of simulation in modern science, this paper argues that R&R studies of simulation are valuable research outputs and should be regarded as an integral and equally important part of scientific progress. R&R should be explicitly addressed and embedded into modelling and simulation practices, and supported by stronger community efforts. Researchers engaging in these efforts face substantial challenges, including those related to recognition and rewards, methodology, and scalability, many of which are under-researched.
help_outline