Constructing MPI Input-output Datatypes for Efficient Transpacking

Communication and file I/O buffers in MPI can contain contiguous as well as non-contiguous, structured data. To describe nonconsecutive data layouts compactly, MPI provides a powerful concept of derived or user-defined datatypes. Especially for MPI-IO, where data are transferred between file and memory buffers, the need for copying between differently typed MPI buffers arise. A straightforward solution to this typed copy problem consists in packing and unpacking the differently structured data via an intermediate buffer. For a maximally efficient MPI(-IO) implementation, means of copying directly (without intermediate buffers) between differently typed MPI buffers are needed. We present a new approach to the typed copy problem. For any two given MPI datatypes describing the layout of input and output buffer, respectively, we show how to efficiently construct an input-output type that subsumes both. This type is used to copy directly from input to output buffer by means of a special transpack function. By completely eliminating the need for intermediate buffering, the typed copy performance can in theory be improved by up to a factor of two, with only a modest overhead for constructing the input-output type. An experimental evaluation shows that even more significant improvements can be achieved in practice.