Sockets Direct Procotol over InfiniBand in clusters: Is it beneficial?

The Sockets Direct Protocol (SDP) has been proposed recently in order to enable sockets based applications to take advantage of the enhanced features provided by, InfiniBand Architecture. In this paper we study the benefits and limitations of an implementation of SDP We first analyze the performance of SDP based on a detailed suite of micro-benchmarks. Next, we evaluate it on two real application domains: (1) A multi-tier Data-Center environment and (2) A Parallel Virtual File System (PVFS). Our microbenchmark results show that SDP is able to provide tip to 2.7 times better bandwidth as compared to the native sockets implementation over InfiniBand (lPoIB) and significantly better latency for large message sizes. Our experimental results also show that SDP is able to achieve a considerably higher performance (improvement of tip to 2.4 times) as compared to lPoIB in the PVFS environment. In the data-center environment, SDP outpeforms IPoIB for large file transfers in-spite of currently being limited by a high connection setup time. However this limitation is entirely implementation specific and as the InfiniBand software and hardware products are rapidly maturing, we expect this limitation to be overcome soon. Based on this, we have shown that the projected performance for SDP, without the connection setup time, can outperform IPoIB for small message transfers as well.