An improved parallel FP-growth algorithm based on Spark and its application

被引：0

作者：

Miao, Yuhang ^{[1
]}

Lin, Jinxing ^{[1
]}

Xu, Nuo ^{[1
]}

机构：

[1] Nanjing Univ Posts & Telecommun, Coll Automat, Nanjing 210023, Peoples R China

来源：

PROCEEDINGS OF THE 38TH CHINESE CONTROL CONFERENCE (CCC) | 2019年

关键词：

Frequent itemset mining; Big data; Parallel FP-growth; Spark; Steam Turbine;

D O I：

10.23919/chicc.2019.8866373

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Frequent itemset mining (FIM) is an important means for data analysis. With the increase of data size, single machine FIM algorithm has the problems of long time-consuming and high memory consumption. Parallel computing of mining algorithm on distributed machine can break through the performance bottleneck of single machine algorithm. In this paper, an improved parallel FP-growth algorithm based on Spark is presented. Firstly, the FP-growth algorithm is improved by matrix technology, compress data set into an information matrix can reduce memory consumption. Then, the improved FP-growth algorithm is parallelized on Spark. Finally, the proposed algorithm is applied to the performance optimization of steam turbine in thermal power plants. The result shows that the proposed algorithm is more efficient than the existing parallel FP-growth algorithm.

引用

页码：3793 / 3797

页数：5

共 50 条

[21] Parallel FP-growth on PC cluster
Pramudiono, I
Kitsuregawa, M
[J]. ADVANCES IN KNOWLEDGE DISCOVERY AND DATA MINING, 2003, 2637 : 467 - 473
[22] TCM Constitution Analysis Method Based on Parallel FP-Growth Algorithm in Hadoop Framework
Li, Mingzheng
Lv, Xiaojuan
Liu, Ye
Wang, Lin
Song, Jianqiang
[J]. JOURNAL OF HEALTHCARE ENGINEERING, 2022, 2022
[23] The Application of FP-Growth Algorithm Based on Distributed Intelligence in Wisdom Medical Treatment
Xu, Fangqin
Lu, Haifeng
[J]. INTERNATIONAL JOURNAL OF PATTERN RECOGNITION AND ARTIFICIAL INTELLIGENCE, 2017, 31 (04)
[24] An Improved Association Rule Mining Algorithm Based on Ant Lion Optimizer Algorithm and FP-Growth
Dong, Dawei
Ye, Zhiwei
Cao, Yu
Xie, Shiwei
Wang, Fengwen
Ming, Wei
[J]. PROCEEDINGS OF THE 2019 10TH IEEE INTERNATIONAL CONFERENCE ON INTELLIGENT DATA ACQUISITION AND ADVANCED COMPUTING SYSTEMS - TECHNOLOGY AND APPLICATIONS (IDAACS), VOL. 1, 2019, : 458 - 463
[25] Scalable algorithm for generation of attribute implication base using FP-growth and spark
Raghavendra Kumar Chunduri
Aswani Kumar Cherukuri
[J]. Soft Computing, 2021, 25 : 9219 - 9240
[26] Scalable algorithm for generation of attribute implication base using FP-growth and spark
Chunduri, Raghavendra Kumar
Cherukuri, Aswani Kumar
[J]. SOFT COMPUTING, 2021, 25 (14) : 9219 - 9240
[27] FP-growth algorithm based on Boolean matrix and MapReduce
College of Computer, Sichuan University, Chengdu 610065, Sichuan, China
[J]. Huanan Ligong Daxue Xuebao, 1 (135-141):
[28] A Power Load Association Rules Mining Method Based on Improved FP-Growth Algorithm
Wang, Ze-Zhong
Cao, Shuo
[J]. 2018 CHINA INTERNATIONAL CONFERENCE ON ELECTRICITY DISTRIBUTION (CICED), 2018, : 2833 - 2837
[29] Fault Diagnosis Technology of Railway Signal Equipment based on Improved FP-Growth Algorithm
Yang, Yueqin
[J]. INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2022, 13 (12) : 664 - 671
[30] PFP: Parallel FP-Growth for Query Recommendation
Li, Haoyuan
Wang, Yi
Zhang, Dong
Zhang, Ming
Chang, Edward Y.
[J]. RECSYS'08: PROCEEDINGS OF THE 2008 ACM CONFERENCE ON RECOMMENDER SYSTEMS, 2008, : 107 - 114

← 1 2 3 4 5 →