An Optimized Distributed OLAP System for Big Data

被引：0

作者：

Chen, Wenhao ^{[1
]}

Wang, Haoxiang ^{[1
]}

Zhang, Xingming ^{[1
]}

Lin, Qidi ^{[1
]}

机构：

[1] South China Univ Technol, Sch Comp Sci & Engn, Guangzhou 510006, Guangdong, Peoples R China

来源：

2017 2ND IEEE INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE AND APPLICATIONS (ICCIA) | 2017年

关键词：

big data; decision making; OLAP;

D O I：

暂无

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

To solve the problems of heterogeneous data types and large amount of calculation in making decision for big data, an optimized distributed OLAP system for big data is proposed in this paper. The system provides data acquisition for different data sources, and supports two types of OLAP engines, Impala and Kylin. First of all, the architecture of the system is proposed, consisting of four modules, data acquisition, data storage, OLAP analysis and data visualization, and the specific implementation of each module is descripted in great detail. Then the optimization of the system is put forward, which is automatic metadata configuration and the cache for OLAP query. Finally, the performance test of the system is conduct to demonstrate that the efficiency of the system is significantly better than the traditional solution.

引用

页码：36 / 40

页数：5

共 50 条

[31] Building a novel physical design of a distributed big data warehouse over a Hadoop cluster to enhance OLAP cube query performance
Ramdane, Yassine
Boussaid, Omar
Boukraa, Doulkifli
Kabachi, Nadia
Bentayeb, Fadila
PARALLEL COMPUTING, 2022, 111
[32] Data Recovery Approach with Optimized Cauchy Coding in Distributed Storage System
Funde, Snehalata
Swain, Gandharba
INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2022, 13 (06) : 620 - 629
[33] Integrating XML data in the TARGIT OLAP system
Pedersen, Torben Bach
Pedersen, Dennis
Pedersen, Jesper
International Journal of Web Engineering and Technology, 2008, 4 (04) : 495 - 533
[34] Integrating XML data in the TARGIT OLAP system
Pedersen, D
Pedersen, J
Pedersen, TB
20TH INTERNATIONAL CONFERENCE ON DATA ENGINEERING, PROCEEDINGS, 2004, : 778 - 781
[35] GDedup: Distributed File System Level Deduplication for Genomic Big Data
Bartus, Paul
Arzuaga, Emmanuel
2018 IEEE INTERNATIONAL CONGRESS ON BIG DATA (IEEE BIGDATA CONGRESS), 2018, : 120 - 127
[36] Computer Performance Determination System Based on Big Data Distributed File
Lu, Kong
CYBER SECURITY INTELLIGENCE AND ANALYTICS, 2020, 928 : 877 - 884
[37] A distributed real-time recommender system for big data streams
Hazem, Heidy
Awad, Ahmed
Yousef, Ahmed Hassan
AIN SHAMS ENGINEERING JOURNAL, 2023, 14 (08)
[38] An approach for Big Data Security based on Hadoop Distributed File system
Mahmoud, Hadeer
Hegazy, Abdelfatah
Khafagy, Mohamed H.
PROCEEDINGS OF 2018 INTERNATIONAL CONFERENCE ON INNOVATIVE TRENDS IN COMPUTER ENGINEERING (ITCE' 2018), 2018, : 109 - 114
[39] HDFSX: Big Data Distributed File System with Small Files Support
EIKafrawy, Passent M.
Sauber, Amr M.
Hafez, Mohamed M.
ICENCO 2016 - 2016 12TH INTERNATIONAL COMPUTER ENGINEERING CONFERENCE (ICENCO) - BOUNDLESS SMART SOCIETIES, 2016, : 131 - 135
[40] Smart Medical Big Data Platform Based on Distributed File System
Cai, Yonghua
BASIC & CLINICAL PHARMACOLOGY & TOXICOLOGY, 2020, 127 : 111 - 111

← 1 2 3 4 5 →