EvoBench: Benchmarking Schema Evolution in NoSQL

Since NoSQL database schema evolution is an important cost and time factor in the development of software applications, a standardized benchmark is essential for the comparison and evaluation of different schema evolution management systems. This benchmark should be simple to be set up, its design and usage ergonomic, its results straightforward for interpretation and reproduction - even after decades. Therefore, we present the implementation of a benchmark using Docker containers. By using containers with databases that already contain the test data, containers with the schema evolution system to be measured and the possibility to use the benchmark system itself in a container, it is very convenient to get the benchmark up and running. We also provide a data generator that creates individual data sets and/or reproduces real data once the schema is known. We demonstrate the flexibility and easy application of our approach by means of several experiments and discuss their results.