An Optimized Distributed OLAP System for Big Data

To solve the problems of heterogeneous data types and large amount of calculation in making decision for big data, an optimized distributed OLAP system for big data is proposed in this paper. The system provides data acquisition for different data sources, and supports two types of OLAP engines, Impala and Kylin. First of all, the architecture of the system is proposed, consisting of four modules, data acquisition, data storage, OLAP analysis and data visualization, and the specific implementation of each module is descripted in great detail. Then the optimization of the system is put forward, which is automatic metadata configuration and the cache for OLAP query. Finally, the performance test of the system is conduct to demonstrate that the efficiency of the system is significantly better than the traditional solution.