Enabling Standard Geospatial Capabilities in Spark for the Efficient Processing of Geospatial Big Data

Nowadays, big data are in the midst of many scientific, economic and societal issues. While most of these data include a spatial component, very few big data processing systems are able to manage this particular component. The authors have assessed the capabilities and limits of current solutions and have concluded that most of them are neither efficient nor extensive enough for spatial data. Furthermore, none of them fully complies with ISO standards and OGC specifications in terms of spatial processing. The authors have sought a way to overcome these limitations and have defined a system in greater accordance with the ISO-19125 standard. The proposed solution, called Elcano, is an extension of Spark complying with ISO-19125, allowing the SQL querying of spatial data and including an original spatial indexation system. Tests demonstrate that Elcano surpasses current available solutions on the market.