Hierarchical Storage for Massive Social Network Data Based on Improved Decision Tree

In order to reasonably store and optimize the read/write (w/r) efficiency of massive social network data, a hierarchical storage method for massive social network data based on improved decision tree is proposed. Improved decision tree is used to classify the social network data into multi-value level. Then the high-level, intermediate-level and low-level data are stored in three different storage containers, namely, memory, SSD and mechanical hard disk. The migration storage strategy with adaptive downgrading and upgrading based on the value-classified data is used to migrate and store the social network data reasonably by combining the data value level and the memory status of the storage containers. In addition, the CFS lock model is used to maintain the w/r integrity of the hierarchical storage. In the experiment, this method can reasonably store data of different value levels. The w/r latency of migration and storage for the social network data is in the range of [1.5ms, 2.1ms], [1.6ms, 2.0ms] respectively, which is extremely short. The packet loss rate is only 0.01%, and the data integrity is high. Meanwhile, it has high throughput and low variance that below 0.15, which can effectively ensure the global consistency of data.