A Bloom Filter-Based Index for Distributed Storage Systems

The indexing technique, which is capable of locating an item, is a key component in distributed storage systems. There have been many solutions for the index in distributed systems. One of the problems is the large number of items and the (relatively) low space available for the index. In this paper we propose a bloom filter based schema for the representation and lookup of items in the distributed systems. In each node, the method selects items and inserts them into a probabilistic data structure. After gathering all the data structures, the index node is in possess of all objects information and is capable of locating items in the system. To reduce the false checking times of the index, we choose items to be recorded in reference with the Internet user behavior pattern. We further use theoretical and experimental analysis to test our proposal. Results show that our method can achieve high performance with limited index space.