Performance improvement by data management layer in a grid RPC system

A grid RPC system provides a useful and intuitive programming interface for master-worker type applications in a grid environment. In many grid applications, such as parameter search programs, both master and workers are often required to have a large amount of common data. Since in the RPC model the data must be transferred from the master directly to each worker, the master is sometimes a bottleneck, resulting in poor performance. In order to improve the performance in such cases, we propose a model to decouple the data transfer by a data management layer from the RPC programming. We have designed and implemented a prototype data transfer layer called OmniStorage to OmniRPC, which is a grid RPC system for parallel programming in a grid environment. This allows efficient data transmission of a large amount of data by placing intermediate relay servers, taking the network topology into account, to route the communication and cache the common data in the server. We have evaluated the performance of the proposed system by using synthetic workloads and a real grid application. The results show that OmniStorage can improve the performance of OmniRPC applications compared to the case of using only OmniRPC.