DiST: A scalable, efficient P2P lookup protocol

A well-known problem found in peer-to-peer systems is how to efficiently and scalably locate the peer that stores a particular data item. In a typical formulation of the problem solution, each data item is mapped to a key; every peer stores data items corresponding to a contiguous range of keys, and locating an item requires identifying the host that holds that item's key. Here we describe Distributed Search Tree (DiST), a distributed lookup protocol based on a straightforward extension of the search tree concept. In DiST peers are assigned to groups, each group is responsible for a range of keys, and groups are located at the nodes of logical search tree. While our approach has comparable complexity to the best algorithms proposed so far (complexity is O(logN)), we show that its flexible design puts it at an advantage when it comes to the application of common performance enhancing techniques such as caching and replication. As an example of such advantage we describe the improvement in data lookup time and resilience obtained with key caching and table lookup replication.