Ecosystem Description of Hadoop Platform Based on HDFS, MapReduce and Data Warehouse Tool Hive

被引：0
|
作者：
Xu, Hongsheng [1 ,2 ]
Chen, Xiangkui [1 ]
Fan, Ganglong [1 ,2 ]
机构：
[1] Luoyang Normal Univ, Luoyang 471934, Peoples R China
[2] Henan Key Lab Big Data Proc & Analyt Elect Commer, Luoyang 471934, Peoples R China
来源：
CYBER SECURITY INTELLIGENCE AND ANALYTICS | 2020年 / 928卷
关键词：
Hadoop; HDFS; MapReduce; Spark; Hive;
D O I：
10.1007/978-3-030-15235-2_149
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
This paper introduces the processing process of the distributed file system (HDFS, MapReduce) which is the core of the Hadoop distributed computing platform and introduces the data warehouse tool Hive and the distributed database Hbase. Spark is a big data distributed programming framework, which not only implements MapReduce operator map function and reduce function and calculation model, but also provides more abundant operators. This paper describes the ecosystem of Hadoop platform based on HDFS, MapReduce and data warehouse tool Hive.
引用
收藏
页码：1127 / 1133
页数：7
相关论文
共 50 条
  • [21] EverAnalyzer: A Self-Adjustable Big Data Management Platform Exploiting the Hadoop Ecosystem
    Karamolegkos, Panagiotis
    Mavrogiorgou, Argyro
    Kiourtis, Athanasios
    Kyriazis, Dimosthenis
    INFORMATION, 2023, 14 (02)
  • [22] Research on university data statistics service platform based on data warehouse
    He Jinlian
    PROCEEDINGS OF THE 2015 4TH NATIONAL CONFERENCE ON ELECTRICAL, ELECTRONICS AND COMPUTER ENGINEERING ( NCEECE 2015), 2016, 47 : 20 - 22
  • [23] Role-Based Access Control Technique with Trino for restriction in Hive-based Data Warehouse
    Georgiev, Angel
    Valkanov, Vladimir
    2024 59TH INTERNATIONAL SCIENTIFIC CONFERENCE ON INFORMATION, COMMUNICATION AND ENERGY SYSTEMS AND TECHNOLOGIES, ICEST 2024, 2024,
  • [24] Research on Distributed Data Mining System Based on Hadoop Platform
    Guo, Jianwei
    Li, Ying
    Du, Liping
    Zhao, Guifen
    Jiang, Jiya
    PROCEEDINGS OF INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND INFORMATION TECHNOLOGY (CSAIT 2013), 2014, 255 : 629 - 636
  • [25] Implementation of power monitoring data cloud platform based on Hadoop
    Du, Jingyi
    Huang, Qiong
    PROCEEDINGS OF 2018 IEEE 3RD ADVANCED INFORMATION TECHNOLOGY, ELECTRONIC AND AUTOMATION CONTROL CONFERENCE (IAEAC 2018), 2018, : 2622 - 2625
  • [26] The calculation and implementation of ARM terminal data based on HADOOP platform
    Zhang, Wei-guo
    Yang, Jia-xiang
    2017 IEEE INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING, COMMUNICATIONS AND COMPUTING (ICSPCC), 2017,
  • [27] A parallel clustering algorithm for Logs Data Based on Hadoop Platform
    Huo, Jiuyuan
    Weng, Jian
    Qu, Hong
    2019 THE 3RD INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE COMPILATION, COMPUTING AND COMMUNICATIONS (HP3C 2019), 2019, : 90 - 94
  • [28] Cost-Based Optimization of Logical Partitions for a Query Workload in a Hadoop Data Warehouse
    Peng, Shu
    Gu, Jun
    Wang, X. Sean
    Rao, Weixiong
    Yang, Min
    Cao, Yu
    WEB TECHNOLOGIES AND APPLICATIONS, APWEB 2014, 2014, 8709 : 559 - 567
  • [29] Developing a big data analytics platform using Apache Hadoop Ecosystem for delivering big data services in libraries
    Singh, Ranjeet Kumar
    DIGITAL LIBRARY PERSPECTIVES, 2024, 40 (02) : 160 - 186
  • [30] A NOVEL MASS METEOROLOGICAL DATA STORAGE SYSTEM BASED ON HADOOP ECOSYSTEM
    Ji, Quanpeng
    FRESENIUS ENVIRONMENTAL BULLETIN, 2021, 30 (05): : 5332 - 5339
12345