Testing feature-rich blockchains

Blockchain implementations have become more and more advanced, combining many different features in the same framework (e.g., oracles, names, and state channels). Since the cost of errors in reputation and represented value is high in the blockchain world, software quality is of the utmost importance. One of the main methods used to assure such high software quality is careful testing. However, the number of tests needed to achieve a high level of assurance grows quadratic with the pairs of features of the blockchain, and when testing triples features the growth is cubic. To manually craft the required large number of tests is an almost impossible undertaking in practice. In this article, we describe how property-based testing (PBT) techniques have been used to automate testing of the core part of the Aeternity blockchain, ensuring the high software quality of the blockchain. Even though PBT is a powerful testing technique, applying it to the task of testing a complex system such as a blockchain, is far from trivial. The structure of the Aeternity property-based test model follows the structure of the blockchain, that is, it cleanly separates different blockchain features (e.g., oracles, smart contracts) into different model parts, and moreover, reduces the amount of boilerplate test model code by focusing on the identification of valid blockchain transactions. The test model is evaluated through a careful instrumentation of test code which permits observations of which combinations of features have been tested during a test run, and with which frequency. This article documents the details of how these issues were addressed in the development of the Aeternity test model, providing insights into both the testing of other blockchains as well as the testing of other complex feature based systems.