Two-Way Trees: A Distributed Algorithm for Efficient Replica Search and Placement

We revisit a distributed caching protocol called random trees for hot spot relieving and low latency in large-scale networks. We propose a new version of random trees called two-way (random) trees that improve the logarithmic lookup path length Theta(log(d) N) to Theta(root log(d) N) with high probability, where d is the degree and N is the size of the tree. Yet, all the optimal properties of random trees are preserved such as balanced workload and the minimum storage requirement without additional communication or computational cost. Two-way trees achieve these by separating trees for lookup and replication. A two-way tree is constructed in a fully distributed and maintenance-free manner, without a-priori known file popularity distributions. We present theoretical models to analyze the maximum workload of any server, and provide provable bounds comparable to the maximum load of a well-known balls into bins problem and a natural queueing model. The experimental results show that two-way trees reduce lookup path length by 60-70% compared to random trees, with minimum increase on server load and the number of replicas created.