What will the future of cloud-based astronomical data processing look like?

被引:0
|
作者
Green, Andrew W. [1 ]
Mannering, Elizabeth [1 ]
Harischandra, Lloyd [1 ]
Minh Vuong [1 ]
O'Toole, Simon [1 ]
Sealey, Katrina [1 ]
Hopkins, Andrew M. [1 ]
机构
[1] Australian Astron Observ, POB 915, N Ryde, NSW 1670, Australia
来源
ASTROINFORMATICS | 2017年 / 12卷 / S325期
关键词
astronomical data bases: miscellaneous; methods: data analysis;
D O I
10.1017/S1743921317001363
中图分类号
P1 [天文学];
学科分类号
0704 ;
摘要
Astronomy is rapidly approaching an impasse: very large datasets require remote or cloud-based parallel processing, yet many astronomers still try to download the data and develop serial code locally. Astronomers understand the need for change, but the hurdles remain high. We are developing a data archive designed from the ground up to simplify and encourage cloud-based parallel processing. While the volume of data we host remains modest by some standards, it is still large enough that download and processing times are measured in days and even weeks. We plan to implement a python based, notebook-like interface that automatically parallelises execution. Our goal is to provide an interface sufficiently familiar and user-friendly that it encourages the astronomer to run their analysis on our system in the cloud-astroinformatics as a service. We describe how our system addresses the approaching impasse in astronomy using the SAMI Galaxy Survey as an example.
引用
收藏
页码:27 / 31
页数:5
相关论文
共 50 条
  • [31] Cloud-Based Data Architecture Security
    Semenov, N. A.
    Poltavtsev, A. A.
    AUTOMATIC CONTROL AND COMPUTER SCIENCES, 2019, 53 (08) : 1056 - 1064
  • [32] Cloud-based backup and data recovery
    Swagatika, Shrabanee
    Panda, Niranjan
    JOURNAL OF INFORMATION & OPTIMIZATION SCIENCES, 2022, 43 (05): : 923 - 932
  • [33] Cardiovascular imaging environment: will the future be cloud-based?
    Kawel-Boehm, Nadine
    Bluemke, David A.
    EXPERT REVIEW OF MEDICAL DEVICES, 2017, 14 (07) : 521 - 528
  • [34] Challenges of Future VANET and Cloud-Based Approaches
    Shrestha, Rakesh
    Bajracharya, Rojeena
    Nam, Seung Yeob
    WIRELESS COMMUNICATIONS & MOBILE COMPUTING, 2018,
  • [35] Data Security in Cloud-Based Applications
    Pandey, Surabhi
    Purohit, G. N.
    Munshi, Usha Mujoo
    DATA SCIENCE LANDSCAPE: TOWARDS RESEARCH STANDARDS AND PROTOCOLS, 2018, 38 : 321 - 326
  • [36] Cloud-based MPC with Encrypted Data
    Alexandru, Andreea B.
    Morari, Manfred
    Pappas, George J.
    2018 IEEE CONFERENCE ON DECISION AND CONTROL (CDC), 2018, : 5014 - 5019
  • [37] Cloud-based data streams optimization
    Najib, Fatma M.
    Ismail, Rasha M.
    Badr, Nagwa L.
    Tolba, Mohamed F.
    WILEY INTERDISCIPLINARY REVIEWS-DATA MINING AND KNOWLEDGE DISCOVERY, 2018, 8 (03)
  • [38] Cloud-based RDF Data Management
    Kaoudi, Zoi
    Manolescu, Ioana
    SIGMOD'14: PROCEEDINGS OF THE 2014 ACM SIGMOD INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA, 2014, : 725 - 729
  • [39] Custody: Towards Data-Aware Resource Sharing in Cloud-Based Big Data Processing
    Ma, Shiyao
    Jiang, Jingjie
    Li, Bo
    Li, Baochun
    2016 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING (CLUSTER), 2016, : 451 - 460
  • [40] What will future vet consults look like?
    Read, Claire
    VETERINARY RECORD, 2020, 187 (06) : 220 - 222