DESIGN AND IMPLEMENTATION OF PARALLEL STATIATICAL ALGORITHM BASED ON HADOOP'S MAPREDUCE MODEL

被引：0
|
作者：
Duan, Songqing [1 ]
Wu, Bin [1 ]
Wang, Bai [1 ]
Yang, Juan [1 ]
机构：
[1] Beijing Univ Posts & Telecommun, Beijing Key Lab Intelligent Telecommun Software &, Beijing, Peoples R China
来源：
2011 IEEE INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTELLIGENCE SYSTEMS | 2011年
基金：
中国国家自然科学基金;
关键词：
Hadoop; MapReduce; Parallel Statistical Algorithm; Central Tendency; Dispersion; Distribution Tendency;
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
The rapid growth of data promotes the development of parallel computing. MapReduce, which is a simplified programming model of distributed parallel computing, is becoming more and more popular. In this paper, we design and implementation of parallel statistical algorithm based on Hadoop ' s MapReduce model. The algorithm, which is used to grasp the overall characteristics of massive data, involves the calculation of central tendency, dispersion and distribution tendency. By experiment, we come to the conclusion that the algorithm is suitable for dealing with large-scale data.
引用
收藏
页码：134 / 138
页数：5
相关论文
共 50 条
  • [1] DESIGN AND IMPLEMENTATION OF PARALLEL TERM CONTRIBUTION ALGORITHM BASED ON MAPREDUCE MODEL
    Chao, Peng
    Bin, Wu
    Chao, Deng
    PROCEEDINGS OF THE 2012 SEVENTH OPEN CIRRUS SUMMIT (OCS 2012), 2012, : 43 - 47
  • [2] Design and implementation of K-means parallel algorithm based on Hadoop
    Jia, Jiyang
    Xie, Hui
    Xu, Tao
    PROCEEDINGS OF 2021 2ND INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND INFORMATION SYSTEMS (ICAIIS '21), 2021,
  • [3] Parallel Implementation of Apriori Algorithm Based on MapReduce
    Li, Ning
    Zeng, Li
    He, Qing
    Shi, Zhongzhi
    INTERNATIONAL JOURNAL OF NETWORKED AND DISTRIBUTED COMPUTING, 2013, 1 (02) : 89 - 96
  • [4] Implementation of Parallel CASINO Algorithm Based on MapReduce
    Zhang, Li
    Shi, Yijie
    PROCEEDINGS OF THE 2016 INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND ENGINEERING APPLICATIONS, 2016, 63 : 104 - 109
  • [5] Parallel implementation of Apriori algorithm based on MapReduce
    Li N.
    Zeng L.
    He Q.
    Shi Z.
    International Journal of Networked and Distributed Computing, 2013, 1 (2) : 89 - 96
  • [6] elephant56: Design and Implementation of a Parallel Genetic Algorithms Framework on Hadoop MapReduce
    Salza, Pasquale
    Ferrucci, Filomena
    Sarro, Federica
    PROCEEDINGS OF THE 2016 GENETIC AND EVOLUTIONARY COMPUTATION CONFERENCE (GECCO'16 COMPANION), 2016, : 1315 - 1322
  • [7] Implementation of Page Rank Algorithm in Hadoop MapReduce Framework
    Bhawivuga, Adhitya
    Kirana, Annisa Puspa
    2016 INTERNATIONAL SEMINAR ON INTELLIGENT TECHNOLOGY AND ITS APPLICATIONS (ISITIA): RECENT TRENDS IN INTELLIGENT COMPUTATIONAL TECHNOLOGIES FOR SUSTAINABLE ENERGY, 2016, : 231 - 235
  • [8] Verification and Validation of Parallel Support Vector Machine Algorithm based on MapReduce Program Model on Hadoop Cluster
    Kiran, M.
    Kumar, Amresh
    Prathap, B. R.
    PROCEEDINGS OF THE 2013 INTERNATIONAL CONFERENCE ON ADVANCED COMPUTING & COMMUNICATION SYSTEMS (ICACCS), 2013,
  • [9] Parallel Reachability Testing Based on Hadoop MapReduce
    Qi, Xiaofang
    Li, Yueran
    SOFTWARE ANALYSIS, TESTING, AND EVOLUTION, SATE 2018, 2018, 11293 : 173 - 184
  • [10] Research on Parallel DBSCAN Algorithm Design Based on MapReduce
    Fu Yan Xiang
    Zhao Wei Zhong
    Ma Hui Fang
    ADVANCED MEASUREMENT AND TEST, PTS 1-3, 2011, 301-303 : 1133 - +
12345