Ecosystem Description of Hadoop Platform Based on HDFS, MapReduce and Data Warehouse Tool Hive

被引：0
|
作者：
Xu, Hongsheng [1 ,2 ]
Chen, Xiangkui [1 ]
Fan, Ganglong [1 ,2 ]
机构：
[1] Luoyang Normal Univ, Luoyang 471934, Peoples R China
[2] Henan Key Lab Big Data Proc & Analyt Elect Commer, Luoyang 471934, Peoples R China
来源：
CYBER SECURITY INTELLIGENCE AND ANALYTICS | 2020年 / 928卷
关键词：
Hadoop; HDFS; MapReduce; Spark; Hive;
D O I：
10.1007/978-3-030-15235-2_149
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
This paper introduces the processing process of the distributed file system (HDFS, MapReduce) which is the core of the Hadoop distributed computing platform and introduces the data warehouse tool Hive and the distributed database Hbase. Spark is a big data distributed programming framework, which not only implements MapReduce operator map function and reduce function and calculation model, but also provides more abundant operators. This paper describes the ecosystem of Hadoop platform based on HDFS, MapReduce and data warehouse tool Hive.
引用
收藏
页码：1127 / 1133
页数：7
相关论文
共 50 条
  • [41] CloudDOE: A User-Friendly Tool for Deploying Hadoop Clouds and Analyzing High-Throughput Sequencing Data with MapReduce
    Chung, Wei-Chun
    Chen, Chien-Chih
    Ho, Jan-Ming
    Lin, Chung-Yen
    Hsu, Wen-Lian
    Wang, Yu-Chun
    Lee, D. T.
    Lai, Feipei
    Huang, Chih-Wei
    Chang, Yu-Jung
    PLOS ONE, 2014, 9 (06):
  • [42] Analysis of Big Data Storage Tools for Data Lakes based on Apache Hadoop Platform
    Belov, Vladimir
    Nikulchev, Evgeny
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2021, 12 (08) : 551 - 557
  • [43] ETL Function Realization of Data Warehouse System Based on SSIS Platform
    Wu, Tong
    2010 2ND INTERNATIONAL WORKSHOP ON DATABASE TECHNOLOGY AND APPLICATIONS PROCEEDINGS (DBTA), 2010,
  • [44] Development of Fault Detection Systems Based on Big Data Ecosystem in Semiconductor Manufacturing: The Hadoop Ecosystem Implementation
    Fu, HuiChu
    Qiao, Yan
    Bai, LiPing
    Wu, NaiQi
    Liu, Bin
    He, YunFang
    IEEE ROBOTICS & AUTOMATION MAGAZINE, 2023, 30 (02) : 22 - 33
  • [45] Design and Implementation of University Information Service Platform Based on Data Warehouse
    Wang, Xiaoguo
    Jia, Ru
    MECHATRONICS AND INDUSTRIAL INFORMATICS, PTS 1-4, 2013, 321-324 : 2543 - 2550
  • [46] An Interface to Heterogeneous Data Sources Based on the Mediator/Wrapper Architecture in the Hadoop Ecosystem
    Schmatz, Klaus-Dieter
    Berwind, Kevin
    Engel, Felix
    Hemmje, Matthias L.
    PROCEEDINGS 2018 IEEE INTERNATIONAL CONFERENCE ON BIOINFORMATICS AND BIOMEDICINE (BIBM), 2018, : 1838 - 1845
  • [47] Replication-Based Query Management for Resource Allocation Using Hadoop and MapReduce over Big Data
    Kumar, Ankit
    Varshney, Neeraj
    Bhatiya, Surbhi
    Singh, Kamred Udham
    BIG DATA MINING AND ANALYTICS, 2023, 6 (04) : 465 - 477
  • [48] Classifying agricultural crop pest data using hadoop MapReduce based C5.0 algorithm
    Revathy R.
    Balamurali S.
    Lawrance R.
    Journal of Cyber Security and Mobility, 2019, 8 (03): : 393 - 408
  • [49] Parallel Fuzzy C-Means Clustering Based Big Data Anonymization Using Hadoop MapReduce
    Lawrance, Josephine Usha
    Jesudhasan, Jesu Vedha Nayahi
    Rittammal, Jerald Beno Thampiraj
    WIRELESS PERSONAL COMMUNICATIONS, 2024, 135 (04) : 2103 - 2130
  • [50] Distributed Case-based Reasoning System Based on Big Data Platform Hadoop
    Wang, Chong-Yang
    Wang, Hong-Bing
    Liang, Yan-Rui
    2015 INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING AND INFORMATION SYSTEM (SEIS 2015), 2015, : 629 - 634
12345