Tiered Storage in Modern Key-Value Stores: Performance, Storage-Efficiency, and Cost-Efficiency Considerations

The emergence of big data has experienced a significant increase in both the variety and volume of data which prompted the development of faster, more reliable storage solutions. The primary objective of this study is to investigate how varying latencies of diverse storage media impact the performance of a representative KVS in RocksDB and to ascertain whether employing lower-cost storage media at the bottom levels can lead to efficient space utilization and cost savings. We implement tiered SSD storage into RocksDB to analyze the trade-offs in employing a diversified setup. Our findings reveal that, despite differences in latency, RocksDB demonstrates remarkable resilience in mitigating the impact of latency variations. Furthermore, to facilitate better storage utilization, we take advantage of RocksDB's flexibility by utilizing different compression techniques in tiered storage while considering their characteristics. Results show that while our strategy does not overcome the performance of Optane SSD, it has a negligible difference with NVMe SSD in terms of write and read throughput. With hybrid compression, compared to SATA SSD, the tiered storage demonstrates a difference of 13.8 MBps in write and 3.86 MBps in read workloads. It also provides a cost-efficient storage option with cost similar to that of SATA SSD but with performance comparable to NVMe SSD. This research offers valuable guidance for organizations aiming to optimize their storage infrastructure and reduce storage upgrade costs while maintaining database efficiency.