DESIGN AND IMPLEMENTATION OF PARALLEL STATIATICAL ALGORITHM BASED ON HADOOP'S MAPREDUCE MODEL

被引：0
|
作者：
Duan, Songqing [1 ]
Wu, Bin [1 ]
Wang, Bai [1 ]
Yang, Juan [1 ]
机构：
[1] Beijing Univ Posts & Telecommun, Beijing Key Lab Intelligent Telecommun Software &, Beijing, Peoples R China
来源：
2011 IEEE INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTELLIGENCE SYSTEMS | 2011年
基金：
中国国家自然科学基金;
关键词：
Hadoop; MapReduce; Parallel Statistical Algorithm; Central Tendency; Dispersion; Distribution Tendency;
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
The rapid growth of data promotes the development of parallel computing. MapReduce, which is a simplified programming model of distributed parallel computing, is becoming more and more popular. In this paper, we design and implementation of parallel statistical algorithm based on Hadoop ' s MapReduce model. The algorithm, which is used to grasp the overall characteristics of massive data, involves the calculation of central tendency, dispersion and distribution tendency. By experiment, we come to the conclusion that the algorithm is suitable for dealing with large-scale data.
引用
收藏
页码：134 / 138
页数：5
相关论文
共 50 条
  • [21] Hadoop MapReduce for Parallel Genetic Algorithm to Solve Traveling Salesman Problem
    Manzi, Entesar
    Bennaceur, Hachemi
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2019, 10 (08) : 97 - 107
  • [22] Parallel Algorithm for indexing large DNA Sequences Using MapReduce on Hadoop
    Kaniwa, Freeson
    Dinakenyane, Otlhapile
    Kuthadi, Venu Madhav
    2017 IEEE INTERNATIONAL CONFERENCE ON BIOINFORMATICS AND BIOMEDICINE (BIBM), 2017, : 1576 - 1582
  • [23] Research and Practice of Distributed Parallel Search Algorithm on Hadoop_MapReduce
    Duan, AiLing
    Si, HaiFang
    2012 INTERNATIONAL CONFERENCE ON CONTROL ENGINEERING AND COMMUNICATION TECHNOLOGY (ICCECT 2012), 2012, : 105 - 108
  • [24] A new data mining algorithm based on MapReduce and hadoop
    Yang, Xianfeng
    Lian, Liming
    International Journal of Signal Processing, Image Processing and Pattern Recognition, 2014, 7 (02) : 131 - 142
  • [25] MapReduce Model of Improved K-Means Clustering Algorithm Using Hadoop MapReduce
    Akthar, Nadeem
    Ahamad, Mohd Vasim
    Ahmad, Shahbaaz
    2016 SECOND INTERNATIONAL CONFERENCE ON COMPUTATIONAL INTELLIGENCE & COMMUNICATION TECHNOLOGY (CICT), 2016, : 192 - 198
  • [26] A Parallel Implementation of Relief Algorithm Using Mapreduce Paradigm
    Yazidi, Jamila
    Bouaguel, Waad
    Essoussi, Nadia
    COMPUTATIONAL COLLECTIVE INTELLIGENCE, ICCCI 2016, PT II, 2016, 9876 : 418 - 425
  • [27] A MapReduce-Based Algorithm for Parallelizing Collusion Detection in Hadoop
    Mortazavi, Mahmood
    Ladani, Behrouz Tork
    2015 7TH CONFERENCE ON INFORMATION AND KNOWLEDGE TECHNOLOGY (IKT), 2015,
  • [28] Multi-pattern Matching Algorithm Based on MapReduce and Hadoop
    Zhang, Wei
    Li, Baolu
    Li, Kun
    PROCEEDINGS 2013 INTERNATIONAL CONFERENCE ON MECHATRONIC SCIENCES, ELECTRIC ENGINEERING AND COMPUTER (MEC), 2013, : 1856 - 1859
  • [29] Massive data MapReduce fingerprint discriminant algorithm Based on Hadoop
    Lu, Wei
    Huang, Jun
    Hong, Lin
    INFORMATION TECHNOLOGY APPLICATIONS IN INDUSTRY, PTS 1-4, 2013, 263-266 : 2655 - +
  • [30] Parallel Implementation of Classification Algorithms Based on MapReduce
    He, Qing
    Zhuang, Fuzhen
    Li, Jincheng
    Shi, Zhongzhi
    ROUGH SET AND KNOWLEDGE TECHNOLOGY (RSKT), 2010, 6401 : 655 - 662
12345