Bristrita: Namespace and Metadata Distribution in Large-Scale Distributed Cloud Storage Systems

The volume of data being processed today in various applications is phenomenally enormous. While the computational speed of the processors to process these data has increased many folds since the last few decades, the storage systems designs to handle the processing of these data do not have adequate features. A crucial entity to support processing of such enormous volume of data is a file system. Until, a few years back, most of the file system designs relied on a single metadata and namespace server and the emergence of distributed metadata and namespace designs is seen only recently. However, most of these distributed namespace designs are based on cluster-based systems where the underlying environment is predictive and can be controlled. In this paper, we present Bristrita, a distributed file system metadata and namespace design layered on top of a peer-to-peer network. Bristrita allows varieties of file system operations, supports files with unlimited size with no limit on the depth of the namespace tree, the number of files in each directory and is resilient to failures. The paper includes the design decisions, the algorithms, and their analysis and shows that it can be used to support users across multiple different geographical regions encompassing a wide area network of computing nodes, storage clusters with support for ad-hoc nodes joining and leaving the network.