An improved parallel FP-growth algorithm based on Spark and its application

被引：0
|
作者：
Miao, Yuhang [1 ]
Lin, Jinxing [1 ]
Xu, Nuo [1 ]
机构：
[1] Nanjing Univ Posts & Telecommun, Coll Automat, Nanjing 210023, Peoples R China
来源：
PROCEEDINGS OF THE 38TH CHINESE CONTROL CONFERENCE (CCC) | 2019年
关键词：
Frequent itemset mining; Big data; Parallel FP-growth; Spark; Steam Turbine;
D O I：
10.23919/chicc.2019.8866373
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
Frequent itemset mining (FIM) is an important means for data analysis. With the increase of data size, single machine FIM algorithm has the problems of long time-consuming and high memory consumption. Parallel computing of mining algorithm on distributed machine can break through the performance bottleneck of single machine algorithm. In this paper, an improved parallel FP-growth algorithm based on Spark is presented. Firstly, the FP-growth algorithm is improved by matrix technology, compress data set into an information matrix can reduce memory consumption. Then, the improved FP-growth algorithm is parallelized on Spark. Finally, the proposed algorithm is applied to the performance optimization of steam turbine in thermal power plants. The result shows that the proposed algorithm is more efficient than the existing parallel FP-growth algorithm.
引用
收藏
页码：3793 / 3797
页数：5
相关论文
共 50 条
  • [31] Construction safety risk analysis of business lines based on improved fp-growth algorithm and Bayesian
    Cai, Jinjin
    Song, Rui
    He, Shiwei
    Zhao, Rixin
    Jiang, Junping
    [J]. Journal of Railway Science and Engineering, 2024, 21 (08) : 3370 - 3381
  • [32] Mining research on correlation factors of residential electricity stability based on improved FP-growth algorithm
    Pan, Hua
    Liu, Rong
    [J]. MANAGEMENT OF ENVIRONMENTAL QUALITY, 2024, 35 (03) : 547 - 566
  • [33] Association Analysis of Complex Product Digital Prototype Data Based on the Improved FP-Growth Algorithm
    Xu, Can
    Liu, Libing
    Tan, Zhihong
    Sun, Shirong
    [J]. 2ND INTERNATIONAL CONFERENCE ON COMPUTER ENGINEERING, INFORMATION SCIENCE AND INTERNET TECHNOLOGY, CII 2017, 2017, : 406 - 415
  • [34] Chinese Document Keyword Extraction Algorithm based on FP-Growth
    Zhao, Meng
    Yu, Wanjun
    Lu, Wenjing
    Liu, Quan
    Li, Jinxiao
    [J]. 2016 INTERNATIONAL CONFERENCE ON SMART CITY AND SYSTEMS ENGINEERING (ICSCSE), 2016, : 202 - 205
  • [35] THE EVALUATION OF ETHNIC COSTUME COURSES BASED ON FP-GROWTH ALGORITHM
    Xu, Rui
    [J]. SCALABLE COMPUTING-PRACTICE AND EXPERIENCE, 2024, 25 (01): : 313 - 326
  • [36] Face Recognition Based on FP-Growth Improved LBP Operator
    Long, Sijie
    Zheng, Chengfeng
    Wu, Canliu
    Qi, Tianyi
    Li, Xiaojie
    Zhu, Yongsheng
    Liu, Jingyi
    Li, Jinwei
    Shuai, Jia
    Xie, Zefeng
    Chen, Zhongyuan
    Du, Hang
    [J]. 2019 3RD INTERNATIONAL CONFERENCE ON DIGITAL SIGNAL PROCESSING (ICDSP 2019), 2019, : 100 - 103
  • [37] CUSTOMIZING FP-GROWTH ALGORITHM TO PARALLEL MINING WITH CHARM plus plus LIBRARY
    Puscian, Marek
    [J]. PHOTONICS APPLICATIONS IN ASTRONOMY, COMMUNICATIONS, INDUSTRY, AND HIGH ENERGY PHYSICS EXPERIMENTS 2017, 2017, 10445
  • [38] Application of Apriori and FP-Growth algorithm in software test data analysis
    Zhou, Mao En
    Liu, Hong
    [J]. 2010 INTERNATIONAL CONFERENCE ON MANAGEMENT SCIENCE AND ENGINEERING (MSE 2010), VOL 5, 2010, : 225 - 228
  • [39] The Research and Improvement Based on FP-Growth Data Mining Algorithm
    Yao, Quanzhu
    Gao, Xingxing
    Lei, Xueli
    Zhang, Tong
    [J]. PROCEEDINGS OF 2016 INTERNATIONAL CONFERENCE ON MODELING, SIMULATION AND OPTIMIZATION TECHNOLOGIES AND APPLICATIONS (MSOTA2016), 2016, 58 : 287 - 293
  • [40] Research on correlation failure models of CRHX EMU traction system based on the improved FP-growth algorithm
    [J]. 2016, Science Press (38):
12345