DESIGN AND IMPLEMENTATION OF PARALLEL STATIATICAL ALGORITHM BASED ON HADOOP'S MAPREDUCE MODEL

被引：0
|
作者：
Duan, Songqing [1 ]
Wu, Bin [1 ]
Wang, Bai [1 ]
Yang, Juan [1 ]
机构：
[1] Beijing Univ Posts & Telecommun, Beijing Key Lab Intelligent Telecommun Software &, Beijing, Peoples R China
来源：
2011 IEEE INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTELLIGENCE SYSTEMS | 2011年
基金：
中国国家自然科学基金;
关键词：
Hadoop; MapReduce; Parallel Statistical Algorithm; Central Tendency; Dispersion; Distribution Tendency;
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
The rapid growth of data promotes the development of parallel computing. MapReduce, which is a simplified programming model of distributed parallel computing, is becoming more and more popular. In this paper, we design and implementation of parallel statistical algorithm based on Hadoop ' s MapReduce model. The algorithm, which is used to grasp the overall characteristics of massive data, involves the calculation of central tendency, dispersion and distribution tendency. By experiment, we come to the conclusion that the algorithm is suitable for dealing with large-scale data.
引用
收藏
页码：134 / 138
页数：5
相关论文
共 50 条
  • [31] Design and Implementation of Parallelized LDA Topic Model Based on MapReduce
    Yan, Duan-wu
    Li, Tie-jun
    Yang, Xiong-fei
    Chen, Kun
    2018 INTERNATIONAL CONFERENCE ON COMPUTER, COMMUNICATION AND NETWORK TECHNOLOGY (CCNT 2018), 2018, 291 : 274 - 278
  • [32] High performance parallel evolutionary algorithm model based on MapReduce framework
    Du, Xin
    Ni, Youcong
    Yao, Zhiqiang
    Xiao, Ruliang
    Xie, Datong
    INTERNATIONAL JOURNAL OF COMPUTER APPLICATIONS IN TECHNOLOGY, 2013, 46 (03) : 290 - 295
  • [33] Parallel Implementation of Chi2 Algorithm in MapReduce Framework
    Zhang, Yong
    Yu, Jingwen
    Wang, Jianying
    HUMAN CENTERED COMPUTING, HCC 2014, 2015, 8944 : 890 - 899
  • [34] MapReduce Implementation for Minimum Reduct Using Parallel Genetic Algorithm
    Alshammari, Mashaan A.
    El-Alfy, El-Sayed M.
    2015 6TH INTERNATIONAL CONFERENCE ON INFORMATION AND COMMUNICATION SYSTEMS (ICICS), 2015, : 13 - 18
  • [35] Performance optimization of MapReduce-based Apriori algorithm on Hadoop cluster
    Singh, Sudhakar
    Garg, Rakhi
    Mishra, P. K.
    COMPUTERS & ELECTRICAL ENGINEERING, 2018, 67 : 348 - 364
  • [36] Genetic Algorithm Based Parallel K-Means Data Clustering Algorithm Using MapReduce Programming Paradigm on Hadoop Environment (GAPKCA)
    Alshammari, Sayer
    Zolkepli, Maslina Binti
    Abdullah, Rusli Bin
    RECENT ADVANCES ON SOFT COMPUTING AND DATA MINING (SCDM 2020), 2020, 978 : 98 - 108
  • [37] Parallel Implementation of K-Means Algorithm Using MapReduce Approach
    Borlea, Ioan-Daniel
    Precup, Radu-Emil
    Dragan, Florin
    Borlea, Alexandra-Bianca
    2018 IEEE 12TH INTERNATIONAL SYMPOSIUM ON APPLIED COMPUTATIONAL INTELLIGENCE AND INFORMATICS (SACI), 2018, : 75 - 80
  • [38] Algorithm implementation and tested of crop growth model based on hadoop of cloud computing
    Jiang, H. (jianghy@njau.edu.cn), 1600, Chinese Society of Agricultural Engineering (29):
  • [39] Implementation of hadoop optimization K-means parallel clustering algorithm
    Huang, Suyu
    Tan, Lingli
    BASIC & CLINICAL PHARMACOLOGY & TOXICOLOGY, 2019, 125 : 160 - 160
  • [40] A MapReduce based Parallel Algorithm for CIM Data Verification
    Liu, Yang
    Shen, Xiaodong
    Xu, Lixiong
    Li, Maozhen
    2014 11TH INTERNATIONAL CONFERENCE ON FUZZY SYSTEMS AND KNOWLEDGE DISCOVERY (FSKD), 2014, : 704 - 709
12345