Ecosystem Description of Hadoop Platform Based on HDFS, MapReduce and Data Warehouse Tool Hive

被引：0
|
作者：
Xu, Hongsheng [1 ,2 ]
Chen, Xiangkui [1 ]
Fan, Ganglong [1 ,2 ]
机构：
[1] Luoyang Normal Univ, Luoyang 471934, Peoples R China
[2] Henan Key Lab Big Data Proc & Analyt Elect Commer, Luoyang 471934, Peoples R China
来源：
CYBER SECURITY INTELLIGENCE AND ANALYTICS | 2020年 / 928卷
关键词：
Hadoop; HDFS; MapReduce; Spark; Hive;
D O I：
10.1007/978-3-030-15235-2_149
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
This paper introduces the processing process of the distributed file system (HDFS, MapReduce) which is the core of the Hadoop distributed computing platform and introduces the data warehouse tool Hive and the distributed database Hbase. Spark is a big data distributed programming framework, which not only implements MapReduce operator map function and reduce function and calculation model, but also provides more abundant operators. This paper describes the ecosystem of Hadoop platform based on HDFS, MapReduce and data warehouse tool Hive.
引用
收藏
页码：1127 / 1133
页数：7
相关论文
共 50 条
  • [1] Hive - A Petabyte Scale Data Warehouse Using Hadoop
    Thusoo, Ashish
    Sen Sarma, Joydeep
    Jain, Namit
    Shao, Zheng
    Chakka, Prasad
    Zhang, Ning
    Antony, Suresh
    Liu, Hao
    Murthy, Raghotham
    26TH INTERNATIONAL CONFERENCE ON DATA ENGINEERING ICDE 2010, 2010, : 996 - 1005
  • [2] A Hadoop/MapReduce based platform for supporting health big data analytics
    Kuo A.
    Chrimes D.
    Qin P.
    Zamani H.
    Studies in Health Technology and Informatics, 2019, 257 : 229 - 235
  • [3] Metadata Storage and Query of Hive Based on Hadoop Distributed Platform
    Xu, Hongsheng
    Wang, Lan
    Fan, Ganglong
    CYBER SECURITY INTELLIGENCE AND ANALYTICS, 2020, 928 : 1079 - 1085
  • [4] Data Warehouse on Hadoop Platform for Decision Support Systems in Education
    Bondarev, Aleksey
    Zakirov, Dilmurat
    2015 TWELVE INTERNATIONAL CONFERENCE ON ELECTRONICS COMPUTER AND COMPUTATION (ICECCO), 2015, : 73 - 76
  • [5] The cooperative study between the hadoop big data platform and the traditional data warehouse
    Hu, Ping
    Open Automation and Control Systems Journal, 2015, 7 (01): : 1144 - 1152
  • [6] Intelligent Service Platform of Manufacturing Process and Tool Based on Data Warehouse
    Wu, Xuefeng
    Feng, Gaocheng
    Wu, Tongkun
    9TH INTERNATIONAL CONFERENCE ON DIGITAL ENTERPRISE TECHNOLOGY - INTELLIGENT MANUFACTURING IN THE KNOWLEDGE ECONOMY ERA, 2016, 56 : 338 - 343
  • [7] A new data mining algorithm based on MapReduce and hadoop
    Yang, Xianfeng
    Lian, Liming
    International Journal of Signal Processing, Image Processing and Pattern Recognition, 2014, 7 (02) : 131 - 142
  • [8] Atrak: a MapReduce-based data warehouse for big data
    Barkhordari, Mohammadhossein
    Niamanesh, Mahdi
    JOURNAL OF SUPERCOMPUTING, 2017, 73 (10): : 4596 - 4610
  • [9] Atrak: a MapReduce-based data warehouse for big data
    Mohammadhossein Barkhordari
    Mahdi Niamanesh
    The Journal of Supercomputing, 2017, 73 : 4596 - 4610
  • [10] Distributed Data Platform System Based on Hadoop Platform
    Guo, Jianwei
    Du, Liping
    Li, Ying
    Zhao, Guifen
    Jiya, Jiang
    PROCEEDINGS OF INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND INFORMATION TECHNOLOGY (CSAIT 2013), 2014, 255 : 533 - 539
12345