Spatial Data Processing with MapReduce

被引:0
|
作者
Gunawardena, Tilani [1 ]
Vicari, Annamaria [2 ]
Mecca, Giansalvatore [1 ]
机构
[1] Univ Basilicata, Dept Math Comp Sci & Econ, Potenza, Italy
[2] Natl Inst Geophys & Volcanol, Rome, Italy
关键词
MapReduce; Spatial Data; GPU computing; CUDA; Hadoop; Big Data;
D O I
暂无
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The current development of high performance parallel supercomputing infrastructures are pushing the boundaries of applications of science and are bringing new paradigms into engineering practices and simulations. Earthquake engineering is also one of the major fields, which benefits from above by looking for solutions in grid computing and cloud computing techniques. Generally, earthquake simulations involve analysis of petabytes of data. Analyzing these large amounts of data in parallel in thousands of nodes in computer clusters results in gaining high performances. Open source cloud solutions such as Hadoop Map Reduce, which is highly scalable and capable of processing large amount of data rapidly in parallel on large clusters provide better solution compared to RDBDM. Both GPUs and MapReduce are designed to support vast data parallelism. For performance considerations, GPU computing could be adopted over low performing CPU systems. This paper discusses MapReduce system using Hadoop and Mars. Mars is a MapReduce framework on graphics processor. Hence, the proposition is to use GPU based systems for earthquake simulations in which Digital elevation model 3D data sets are fully materialized where scientist can make use of these data for various analysis and simulations.
引用
收藏
页码:485 / 490
页数:6
相关论文
共 50 条
  • [1] Experiences on Processing Spatial Data with MapReduce
    Cary, Ariel
    Sun, Zhengguo
    Hristidis, Vagelis
    Rishe, Naphtali
    [J]. SCIENTIFIC AND STATISTICAL DATABASE MANAGEMENT, PROCEEDINGS, 2009, 5566 : 302 - 319
  • [2] Simplifying MapReduce data processing
    Liao, Chih-Shan
    Shih, Jin-Ming
    Chang, Ruay-Shiung
    [J]. INTERNATIONAL JOURNAL OF COMPUTATIONAL SCIENCE AND ENGINEERING, 2013, 8 (03) : 219 - 226
  • [3] Spatial Data Analysis with ArcGIS and MapReduce
    Singh, Hari
    Bawa, Seema
    [J]. 2016 IEEE INTERNATIONAL CONFERENCE ON COMPUTING, COMMUNICATION AND AUTOMATION (ICCCA), 2016, : 45 - 49
  • [4] SpatialHadoop: A MapReduce Framework for Spatial Data
    Eldawy, Ahmed
    Mokbel, Mohamed F.
    [J]. 2015 IEEE 31ST INTERNATIONAL CONFERENCE ON DATA ENGINEERING (ICDE), 2015, : 1352 - 1363
  • [5] Improving the performance of GIS polygon overlay computation with MapReduce for spatial big data processing
    Yong Wang
    Zhenling Liu
    Hongyan Liao
    Chengjun Li
    [J]. Cluster Computing, 2015, 18 : 507 - 516
  • [6] MapReduce: A Flexible Data Processing Tool
    Dean, Jeffrey
    Ghemawat, Sanjay
    [J]. COMMUNICATIONS OF THE ACM, 2010, 53 (01) : 72 - 77
  • [7] Improving the performance of GIS polygon overlay computation with MapReduce for spatial big data processing
    Wang, Yong
    Liu, Zhenling
    Liao, Hongyan
    Li, Chengjun
    [J]. CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS, 2015, 18 (02): : 507 - 516
  • [8] Prominence of MapReduce in BIG DATA Processing
    Pandey, Shweta
    Tokekar, Vrinda
    [J]. 2014 FOURTH INTERNATIONAL CONFERENCE ON COMMUNICATION SYSTEMS AND NETWORK TECHNOLOGIES (CSNT), 2014, : 555 - 560
  • [9] Parallel Data Processing with MapReduce: A Survey
    Lee, Kyong-Ha
    Lee, Yoon-Joon
    Choi, Hyunsik
    Chung, Yon Dohn
    Moon, Bongki
    [J]. SIGMOD RECORD, 2011, 40 (04) : 11 - 20
  • [10] MapReduce: Simplified data processing on large clusters
    Dean, J
    Ghemawat, S
    [J]. USENIX ASSOCIATION PROCEEDINGS OF THE SIXTH SYMPOSIUM ON OPERATING SYSTEMS DESIGN AND IMPLEMENTATION (OSDE '04), 2004, : 137 - 149