Architecture of Processing and Analysis Systems for Big Astronomical Data

被引：0
|
作者：
Kolosov, Ivan [1 ]
Gerasimov, Sergey [1 ]
Meshcheryakov, Alexander [2 ,3 ]
机构：
[1] Lomonosov Moscow State Univ, Fac Computat Math & Cybernet, Moscow, Russia
[2] Russian Acad Sci, Space Res Inst, Profsoyuznaya 84-3, Moscow 117997, Russia
[3] Kazan Fed Univ, Kremlevskaya Str 18, Kazan 420008, Russia
来源：
ASTRONOMICAL DATA ANALYSIS SOFTWARE AND SYSTEMS XXVI | 2019年 / 521卷
基金：
俄罗斯基础研究基金会;
关键词：
D O I：
暂无
中图分类号：
P1 [天文学];
学科分类号：
0704 ;
摘要：
This work explores the use of big data technologies deployed in the cloud for processing of astronomical data. We have applied Hadoop and Spark to the task of co-adding astronomical images. We compared the overhead and execution time of these frameworks. We conclude that the performance of both frameworks is generally on par. The Spark API is more flexible, which allows one to easily construct astronomical data processing pipelines.
引用
收藏
页码：428 / 430
页数：3
相关论文
共 50 条
  • [21] Big Data application on signal processing systems
    Tikhonyuk, A. I.
    Erokhin, S. D.
    Chadov, T. A.
    2018 SYSTEMS OF SIGNAL SYNCHRONIZATION, GENERATING AND PROCESSING IN TELECOMMUNICATIONS (SYNCHROINFO), 2018,
  • [22] Research of Configurable Hybrid Memory Architecture for Big Data Processing
    Zhou, Hongwei
    Deng, Rangyu
    Feng, Quanyou
    Ni, Xiaoqiang
    Dou, Qiang
    COMPUTER ENGINEERING AND TECHNOLOGY, NCCET 2017, 2018, 600 : 116 - 132
  • [23] System Architecture for Processing and Storing Big Data in industrial Applications
    Kleegrewe, Christian
    Dippl, Sebastian
    AUTOMATION 2013, 2013, 2209 : 11 - 14
  • [24] A modular software architecture for processing of big geospatial data in the cloud
    Kraemer, Michel
    Senner, Julia
    COMPUTERS & GRAPHICS-UK, 2015, 49 : 69 - 81
  • [25] Fog Computing Architecture for Scalable Processing of Geospatial Big Data
    Barik, Rabindra K.
    Priyadarshini, Rojalina
    Lenka, Rakesh K.
    Dubey, Harishchandra
    Mankodiya, Kunal
    INTERNATIONAL JOURNAL OF APPLIED GEOSPATIAL RESEARCH, 2020, 11 (01) : 1 - 20
  • [26] An Architecture for Healthcare Big Data Management and Analysis
    Gui, Hao
    Zheng, Rong
    Ma, Chao
    Fan, Hao
    Xu, Liya
    HEALTH INFORMATION SCIENCE, HIS 2016, 2016, 10038 : 154 - 160
  • [27] Simplifying Big Data Analytics Systems with a Reference Architecture
    Sang, Go Muan
    Xu, Lai
    de Vrieze, Paul
    COLLABORATION IN A DATA-RICH WORLD, 2017, 506 : 242 - 249
  • [28] Big Data Architecture for Building Energy Management Systems
    Ruiz, M. Dolores
    Gomez-Romero, Juan
    Fernandez-Basso, Carlos
    Martin-Bautista, Maria J.
    IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, 2022, 18 (09) : 5738 - 5747
  • [29] ENTERPRISE SERVICE BUS ARCHITECTURE FOR THE BIG DATA SYSTEMS
    Orlowski, Cezary
    Szczerbicki, Edward
    Grabowski, Jan
    MANAGEMENT AND PRODUCTION ENGINEERING REVIEW, 2014, 5 (01) : 28 - 31
  • [30] Correction to: Big Data Processing and Analysis in Internet of Vehicles: Architecture, Taxonomy, and Open Research Challenges
    Ansif Arooj
    Muhammad Shoaib Farooq
    Aftab Akram
    Razi Iqbal
    Ashutosh Sharma
    Gaurav Dhiman
    Archives of Computational Methods in Engineering, 2022, 29 : 731 - 731
12345