DESIGN AND IMPLEMENTATION OF PARALLEL STATIATICAL ALGORITHM BASED ON HADOOP'S MAPREDUCE MODEL

被引：0
|
作者：
Duan, Songqing [1 ]
Wu, Bin [1 ]
Wang, Bai [1 ]
Yang, Juan [1 ]
机构：
[1] Beijing Univ Posts & Telecommun, Beijing Key Lab Intelligent Telecommun Software &, Beijing, Peoples R China
来源：
2011 IEEE INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTELLIGENCE SYSTEMS | 2011年
基金：
中国国家自然科学基金;
关键词：
Hadoop; MapReduce; Parallel Statistical Algorithm; Central Tendency; Dispersion; Distribution Tendency;
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
The rapid growth of data promotes the development of parallel computing. MapReduce, which is a simplified programming model of distributed parallel computing, is becoming more and more popular. In this paper, we design and implementation of parallel statistical algorithm based on Hadoop ' s MapReduce model. The algorithm, which is used to grasp the overall characteristics of massive data, involves the calculation of central tendency, dispersion and distribution tendency. By experiment, we come to the conclusion that the algorithm is suitable for dealing with large-scale data.
引用
收藏
页码：134 / 138
页数：5
相关论文
共 50 条
  • [41] Parallel k-modes Algorithm based on MapReduce
    Guo Tao
    Ding Xiangwu
    Li Yefeng
    2015 THIRD INTERNATIONAL CONFERENCE ON DIGITAL INFORMATION, NETWORKING, AND WIRELESS COMMUNICATIONS (DINWC), 2015, : 176 - 179
  • [42] The network data parallel parsing algorithm based on MapReduce
    Zhang Huimin
    Feng Linsheng
    2017 SECOND INTERNATIONAL CONFERENCE ON MECHANICAL, CONTROL AND COMPUTER ENGINEERING (ICMCCE), 2017, : 203 - 207
  • [43] A MapReduce based Parallel Algorithm for CIM Data Verification
    Liu, Yang
    Shen, Xiaodong
    Xu, Lixiong
    Li, Maozhen
    2014 11TH INTERNATIONAL CONFERENCE ON FUZZY SYSTEMS AND KNOWLEDGE DISCOVERY (FSKD), 2014, : 745 - 750
  • [44] Parallel Diffrential Evolution Clustering Algorithm based on MapReduce
    Daoudi, Meroua
    Hamena, Soumiya
    Benmounah, Zakaria
    Batouche, Mohamed
    2014 6TH INTERNATIONAL CONFERENCE OF SOFT COMPUTING AND PATTERN RECOGNITION (SOCPAR), 2014, : 337 - 341
  • [45] PARALLEL COREGISTRATION ALGORITHM FOR SAR IMAGES BASED ON HADOOP
    Li, Jiawei
    Zeng, Guobing
    Xu, Huaping
    IGARSS 2023 - 2023 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM, 2023, : 7957 - 7960
  • [46] Research of parallel DBSCAN clustering algorithm based on MapReduce
    Fu, X. (xffu@gdut.edu.cn), 1600, Science and Engineering Research Support Society (07):
  • [47] Design and Implementation of Recommender System Based on Hadoop
    Wang, Qing
    PROCEEDINGS OF 2016 IEEE 7TH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING AND SERVICE SCIENCE (ICSESS 2016), 2016, : 295 - 299
  • [48] A parallel algorithm of public bus accounting based on MapReduce
    Wang, S., 1600, Asian Network for Scientific Information (12):
  • [49] Network Intrusion Detection with a Hashing Based Apriori Algorithm Using Hadoop MapReduce
    Azeez, Nureni Ayofe
    Ayemobola, Tolulope Jide
    Misra, Sanjay
    Maskeliunas, Rytis
    Damasevicius, Robertas
    COMPUTERS, 2019, 8 (04)
  • [50] Research on PageRank Algorithm parallel computing Based on Hadoop
    Yang, Pengfei
    Zhou, Liqing
    Proceedings of the 2016 4th International Conference on Mechanical Materials and Manufacturing Engineering (MMME 2016), 2016, 79 : 182 - 185
12345