Complex Simulation Workflows in Containerized High-Performance Environment

Cutting-edge research involving in-silico simulations often requires to use many heterogeneous software tools made by different developers, resulting in complex, custom-made pipelines of various scripts and programs. Such pipelines are nearly impossible to be reproduced by other research groups, jeopardizing both quality and acceptance of such research results. Starting with two in-house use cases in computational chemistry, we identified a common pattern applicable for other applications as well, and we designed and implemented a solution based on Jupyter notebooks to drive the simulation, Docker containers to package all software dependencies, and Kubernetes execution environment to run several cooperating containers which build up the whole application.