Automatic Hybrid OpenMP plus MPI Program Generation for Dynamic Programming Problems

We describe a program that automatically generates a hybrid OpenMP + MPI program for a class of recursive calculations with template dependencies. Many useful generalized dynamic programming problems fit this category, such as Multiple String Alignment and multi-arm Bernoulli Bandit problems. Solving problems like these, especially those involving several dimensions, can use a significant amount of memory and time. Our generator addresses these issues by dividing the problem into many tiles that can be solved in parallel. Programs generated using this program generator are capable of solving large problems and achieve good scalability when run on many cores. The input supplied to the generator is a high level description of the problem; the output is a fully functioning parallel program for a cluster of shared memory nodes. This high level approach to parallel computation allows the generator to have a large amount of control over memory allocation, load balancing and calculation ordering.