Comparative Analysis of Apache Spark and Hadoop MapReduce Using Various Parameters and Execution Time

被引：1

作者：

Meena, Bhagavathula ^{[1
]}

Sarwani, I. S. L. ^{[2
]}

Archana, M. ^{[3
]}

Supriya, P. ^{[4
]}

机构：

[1] Raghu Engn Coll, CSE Dept, Visakhapatnam, Andhra Pradesh, India

[2] ANITS, Visakhapatnam, Andhra Pradesh, India

[3] CVR Coll Engn, Hyderabad, Telangana, India

[4] Raghu Engn Coll, Visakhapatnam, Andhra Pradesh, India

来源：

INTELLIGENT COMPUTING AND COMMUNICATION, ICICC 2019 | 2020年 / 1034卷

关键词：

Hadoop; Apache Spark; Big Data; HDFS; MapReduce;

D O I：

10.1007/978-981-15-1084-7_70

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Due to rapid growth in Information technology there is a lot of advancement in Electronics and communication. Every hour lot of data with various medium is getting generated which is referred as big data. Big Data and Hadoop are the trending terms nowadays. Storage and analysis of such a large data is becoming one of the challenges for computer science and Information Technology devotee throughout the world in the most recent couple of the years. As Apache Spark and Hadoop are the frameworks used for analyzing big data, our paper discusses a comparison of both the frame works by choosing different sizes of datasets and in terms of time comparison also. This comparison is made using word count algorithm. Although both the resources are relayed on an idea of significantly varying Big Data performance. This paper shows an analysis on both frameworks for word count algorithm over Hadoop MapReduce and Apache spark environment

引用

页码：719 / 725

页数：7

共 50 条

[41] Typhoon Quantitative Rainfall Prediction from Big Data Analytics by Using the Apache Hadoop Spark Parallel Computing Framework
Wei, Chih-Chiang
Chou, Tzu-Hao
[J]. ATMOSPHERE, 2020, 11 (08)
[42] STREAM TEXT DATA ANALYSIS ON TWITTER USING APACHE SPARK STREAMING
Hakdagli, Ozlem
Ozcan, Caner
Ogul, Iskender Ulgen
[J]. 2018 26TH SIGNAL PROCESSING AND COMMUNICATIONS APPLICATIONS CONFERENCE (SIU), 2018,
[43] A Big Data Analysis Framework Using Apache Spark and Deep Learning
Gupta, Anand
Thakur, Hardeo Kumar
Shrivastava, Ritvik
Kumar, Pulkit
Nag, Sreyashi
[J]. 2017 17TH IEEE INTERNATIONAL CONFERENCE ON DATA MINING WORKSHOPS (ICDMW 2017), 2017, : 9 - 16
[44] Stock Market Real Time Recommender Model Using Apache Spark Framework
Seif, Mostafa Mohamed
Hamed, Essam M. Ramzy
Hegazy, Abd El Fatah Abdel Ghfar
[J]. INTERNATIONAL CONFERENCE ON ADVANCED MACHINE LEARNING TECHNOLOGIES AND APPLICATIONS (AMLTA2018), 2018, 723 : 671 - 683
[45] Performance Analysis of Network Intrusion Detection Schemes using Apache Spark
Kulariya, Manish
Saraf, Priyanka
Ranjan, Raushan
Gupta, Govind P.
[J]. 2016 INTERNATIONAL CONFERENCE ON COMMUNICATION AND SIGNAL PROCESSING (ICCSP), VOL. 1, 2016, : 1973 - 1977
[46] Distributed Pattern Matching and Document Analysis in Big Data using Hadoop MapReduce Model
Ramya, A., V
Sivasankar, E.
[J]. 2014 INTERNATIONAL CONFERENCE ON PARALLEL, DISTRIBUTED AND GRID COMPUTING (PDGC), 2014, : 312 - 317
[47] Leveraging Hadoop Framework to develop Duplication Detector and analysis using MapReduce, Hive and Pig
Sethi, Priyanka
Kumar, Prakash
[J]. 2014 SEVENTH INTERNATIONAL CONFERENCE ON CONTEMPORARY COMPUTING (IC3), 2014, : 454 - 460
[48] Real-time Processing of IoT Events with Historic data using Apache Kafka and Apache Spark with Dashing framework
D'silva, Godson Michael
Khan, Azharuddin
Joshi, Gaurav
SiddheshBari
[J]. 2017 2ND IEEE INTERNATIONAL CONFERENCE ON RECENT TRENDS IN ELECTRONICS, INFORMATION & COMMUNICATION TECHNOLOGY (RTEICT), 2017, : 1804 - 1809
[49] Resilient Distributed Computing Platforms for Big Data Analysis Using Spark and Hadoop
Chang, Bao Rong
Tsai, Hsiu-Fen
Wang, Yo-Ai
Huang, Chien-Feng
[J]. PROCEEDINGS OF 2016 INTERNATIONAL CONFERENCE ON APPLIED SYSTEM INNOVATION (ICASI), 2016,
[50] Real-Time Heart Arrhythmia Detection Using Apache Spark Structured Streaming
Ilbeigipour, Sadegh
Albadvi, Amir
Akhondzadeh Noughabi, Elham
[J]. JOURNAL OF HEALTHCARE ENGINEERING, 2021, 2021

← 1 2 3 4 5 →