Reproducibility in M&S research: issues, strategies and implications for model development environments

As the use of computer simulation is increasingly becoming central to scientific enterprise, lack of proper documentation, validation and distribution of models and experiments may hamper reproducibility and hence cause a credibility gap. Building on prior research requires ability to reproduce simulation experiments and to further extend models to address increasingly challenging problems. In this article, we delineate three dimensions of reproducible research. For each dimension, issues, strategies and implications for simulation model development are identified. From a technical infrastructure point of view, we introduce the e-Portfolio concept, which is an ensemble of integrated active documents that interweave published manuscript, computer code, data and scientific workflow specification. From the process point of view, strategies and potential mechanisms are delineated to enable authors, publishers, funding agencies, journals and the broader scientific community to cooperate and establish a sustained model base, simulations, experiments and documentation, so that scientists can build on each other's work and achievements.