From toys to Teraflops: Bridging the Beowulf gap

Do-it-yourself supercomputing has emerged as a solution to cost-effectively sustain the computational demands of the scientific research community. Despite some of the successes of this approach, represented by Beowulf-class computing, it has limitations that need to be recognized as well as problems that need to be resolved in order to extend its scope of applicability. While the performance of hardware incorporated into these systems has continued to improve at a remarkable rate, enabling the execution of steadily larger and more compute-intensive applications, the software environment of the machines has seen little to no improvement or evolution. The authors find that this gap between the rates of development of hardware and software is a crucial obstacle to the full exploitation of these systems into and beyond the Teraflops realm. They provide suggestions as to how this gap might be narrowed within the context of Beowulf-class computing.