Towards Lightweight and Swift Storage Resource Management in Big Data Cloud Era

Workload IO behavior in modern data centers is fluctuating and unpredictable due to the rapidly adopted, public cloud environment. Nevertheless, existing storage resource management systems, such as VMware SDRS, are incapable of performing real time policy-based storage management due to the high cost of migrating large size virtual disks. Hence, the traditional storage management schemes become ineffective due to the lack of quick response to the frequent IO bursts and the inaccurate storage latency prediction in the light of a highly fluctuating environment. To address the aforementioned issues, we propose LightSRM, which can work properly in a time-variant cloud environment. To mitigate the storage migration cost, we leverage copy-on-write/read snapshots to redirect the IO requests without moving the virtual disk. To support snapshots in storage management, we also build a performance model specifically for snapshots. We employ exponentially weighted moving average with adjustable sliding window to provide quick and accurate performance prediction. Furthermore, we propose a hybrid management scheme, which can dynamically choose either snapshot or migration for fastest performance tuning. We build our prototype in a QEMU/KVM based virtualized environment. Our empirical evaluation results show that snapshot can redirect IO requests in a faster manner than migration can do when the virtual disk size is large. Besides, snapshot method has less disk performance impact on the applications. By employing hybrid snapshot/migration method, LightSRM yields less overall latency, better load balance, and less IO traffic overhead.