STRATEGIES FOR SIMULATION SOFTWARE QUALITY ASSURANCE APPLIED TO OPEN SOURCE DEM - PARTICLES 2011

We present a strategy to improve the software quality for scientific simulation software, applied to the open source DEM code LIGGGHTS [1] [2]. We aim to improve the quality of the LIGGGHTS DEM code by two measures: Firstly, making the simulation code open source gives the whole user community the possibility to detect bugs in the source code and make suggestions to improve the code quality. Secondly, we apply a test harness, which is an important part of the work-flow for quality assurance in software engineering [5]. In the case of scientific simulation software, it consists of a set of simulation examples that should span the range of applicability of the software as good as possible. Technically, in our case it consists of a set of 10-50 LIGGGHTS simulations and is being run automatically on our cluster, where the number of processors, the code features and the numerical models are varied. Qualitative results are automatically extracted and are plotted for comparison, so thus a huge parameter space of flow regimes, numerical models, code features and parallelization situations can be governed. A test harness can aid in (a) finding bugs in the software, (b) checking parallel efficiency and consistency, (c) comparing different numerical models, and, most importantly, (d) experimental validation. Parallel consistency means that within a parallel framework, we need to have the possibility to compare the answers that a run with a different number of processors gives and the time that it takes to compute them. Experimental validation is especially important for scientific simulations. If experimental data is available for a test case, the experimental data is automatically compared to the numerical results, by means of global quantities such number of particles in the simulation, translational and rotational kinetic energy, thermal energy etc. The LIGGGHTS test harness aims to be a transparent and open community effort that everybody can contribute to in order to improve the quality of the LIGGGHTS code. We illustrate the usefulness of the test harness with several examples, where we especially focus on experimental validation.