Architecture of Geospatial Big-Data Batch Processing Model Based on Hadoop

被引:0
|
作者
Kim, Sang-Su [1 ]
Yu, Sung-Hwan [1 ]
机构
[1] ICTWAY Corp, IC Res Lab, Dae Jeon, South Korea
关键词
Big-Data; Spatial Hadoop; Spatial ETL; Apache Storm; Apache Hadoop;
D O I
暂无
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In this paper, we propose a spatial ETL toolkit model, regardless of collectible data format, to replace traditional ETL (Extract, Transform, Load) to process public or individual system server with fast data creation. Recently various web servers, and the need of analyzing structured/unstructured Big-Data log information generated by independent high-speed servers are increasing. In addition, the space ETL model for replacing an existing service, which is built into a relational database is required and replaced the existing system in order to analyze unstructured data log information in Big-Data environments, where space is needed the ETL model. This paper proposes the spatial ETL model and proposed model is possible loading log information regardless of connected Hadoop or local storage to input data format quickly and accurately. Existing relational database consist of structured data is possible connection and extend function.
引用
收藏
页码:964 / 966
页数:3
相关论文
共 50 条
  • [1] Data Modifications in Blockchain Architecture for Big-Data Processing
    Tulkinbekov, Khikmatullo
    Kim, Deok-Hwan
    [J]. SENSORS, 2023, 23 (21)
  • [2] Implementation of a Distributed Processing Engine for Spatial Big-Data Processing based on Batch and Stream
    Kim, Sang-Su
    Song, Kwaun-Sik
    [J]. 2017 INTERNATIONAL CONFERENCE ON INFORMATION AND COMMUNICATION TECHNOLOGY CONVERGENCE (ICTC), 2017, : 1196 - 1198
  • [3] Design of big data processing system architecture based on Hadoop Under the cloud computing
    Duan, Chunmei
    [J]. MECHATRONICS ENGINEERING, COMPUTING AND INFORMATION TECHNOLOGY, 2014, 556-562 : 6302 - 6306
  • [4] A Study for Big-Data (Hadoop) Application in Semiconductor Manufacturing
    Kang, Sheng
    Chien, Wei-Ting Kary
    Yang, Jun Gang
    [J]. 2016 IEEE INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND ENGINEERING MANAGEMENT (IEEM), 2016, : 1893 - 1897
  • [5] Fog Computing Architecture for Scalable Processing of Geospatial Big Data
    Barik, Rabindra K.
    Priyadarshini, Rojalina
    Lenka, Rakesh K.
    Dubey, Harishchandra
    Mankodiya, Kunal
    [J]. INTERNATIONAL JOURNAL OF APPLIED GEOSPATIAL RESEARCH, 2020, 11 (01) : 1 - 20
  • [6] A modular software architecture for processing of big geospatial data in the cloud
    Kraemer, Michel
    Senner, Julia
    [J]. COMPUTERS & GRAPHICS-UK, 2015, 49 : 69 - 81
  • [7] The Research on Big Data Security Architecture Based on Hadoop
    Zhuang, Miao
    [J]. PROCEEDINGS OF THE 2015 4TH NATIONAL CONFERENCE ON ELECTRICAL, ELECTRONICS AND COMPUTER ENGINEERING ( NCEECE 2015), 2016, 47 : 241 - 244
  • [8] Big medical data processing system based on hadoop
    Liu, W.
    [J]. BASIC & CLINICAL PHARMACOLOGY & TOXICOLOGY, 2020, 126 : 181 - 181
  • [9] Use of Big-Data in healthcare and lifescience using Hadoop technologies
    Zaveri, Chintan
    [J]. PROCEEDINGS OF THE 2017 IEEE SECOND INTERNATIONAL CONFERENCE ON ELECTRICAL, COMPUTER AND COMMUNICATION TECHNOLOGIES (ICECCT), 2017,
  • [10] A Systematic Review On Cardiovascular diseases using Big-Data by Hadoop
    Thakur, Sanjeev
    Ramzan, Munaza
    [J]. 2016 6TH INTERNATIONAL CONFERENCE - CLOUD SYSTEM AND BIG DATA ENGINEERING (CONFLUENCE), 2016, : 351 - 355