Cookery: A framework for creating data processing pipeline using online services

被引:0
|
作者
Baranowski M. [1 ]
Belloum A. [1 ,2 ]
Cushing R. [1 ]
Valkering O. [1 ]
机构
[1] Multiscale Networked Systems (MNS), Institute of Informatics University of Amsterdam, Amsterdam
[2] Netherlands eScience Center Science, Park 140, Amsterdam
来源
| 1600年 / Slovak Academy of Sciences卷 / 39期
基金
欧盟地平线“2020”;
关键词
AWS Lambda; Domain specific languages (DSL); Function-as-a-service (FaaS); Serverless computing;
D O I
10.31577/CAI_2020_4_678
中图分类号
学科分类号
摘要
With the increasing amount of data the importance of data analysis in various scientific domains has grown. A large amount of the scientific data has shifted to cloud based storage. The cloud offers storage and computation power. The Cookery framework is a tool developed to build scientific applications using cloud services. In this paper we present the Cookery systems and how they can be used to authenticate and use standard online third party services to easily create data analytic pipelines. Cookery framework is not limited to work with standard web services; it can also integrate and work with the emerging AWS Lambda which is part of a new computing paradigm, collectively, known as serverless computing. The combination of AWS Lambda and Cookery, which makes it possible for users in many scientific domains, who do not have any program experience, to create data processing pipelines using cloud services in a short time. © 2020 Slovak Academy of Sciences. All rights reserved.
引用
收藏
页码:678 / 694
页数:16
相关论文
共 50 条
  • [1] Cookery: A Framework for creating data processing pipeline using online services
    Branowski, Mikolaj
    Belloum, Adam
    2018 IEEE 14TH INTERNATIONAL CONFERENCE ON E-SCIENCE (E-SCIENCE 2018), 2018, : 368 - 369
  • [2] COOKERY: A FRAMEWORK FOR CREATING DATA PROCESSING PIPELINE USING ONLINE SERVICES
    Baranowski, Mikolaj
    Belloum, Adam
    Cushing, Reginald
    Valkering, Onno
    COMPUTING AND INFORMATICS, 2020, 39 (04) : 678 - 694
  • [3] Rapid static GNSS data processing using online services
    Berber, M.
    Ustun, A.
    Yetkin, M.
    JOURNAL OF GEODETIC SCIENCE, 2014, 4 (01) : 123 - 129
  • [4] A scalable pipeline data processing framework using database and visualization techniques
    Han, Wook-Shin
    Jung, Soon Ki
    Shin, Jeyong
    Lee, Jinsoo
    Yoon, Mina
    Yoon, Chang Geol
    Seo, Won Scok
    Koo, Sang Ok
    ADVANCED INTELLIGENT COMPUTING THEORIES AND APPLICATIONS: WITH ASPECTS OF THEORETICAL AND METHODOLOGICAL ISSUES, 2007, 4681 : 334 - +
  • [5] Creating a Robust Online Pipeline
    Baena, Carlos
    SIGGRAPH '19 -ACM SIGGRAPH 2019 TALKS, 2019,
  • [6] GPS Processing Using Online Services
    Tsakiri, M.
    JOURNAL OF SURVEYING ENGINEERING-ASCE, 2008, 134 (04): : 115 - 125
  • [7] Creating a framework for online cancer services research to facilitate timely and interdisciplinary applications
    Whitten, P
    Kreps, GL
    Eastin, MS
    JOURNAL OF MEDICAL INTERNET RESEARCH, 2005, 7 (03)
  • [8] TaskRouter: A Newly Designed Online Data Processing Framework
    Gu, Minhao
    Zhu, Kejun
    Li, Fei
    Shen, Wei
    2016 IEEE-NPSS REAL TIME CONFERENCE (RT), 2016,
  • [9] An efficient data analysis framework for online security processing
    Li, Jun
    Liu, Yanzhao
    Liu, Yanzhao (liuyz@itsec.gov.cn), 2021, Hindawi Limited (2021)
  • [10] A New Big Data Processing Framework for the Online Roadshow
    Leow, Kang-Ren
    Leow, Meng-Chew
    Ong, Lee-Yeng
    BIG DATA AND COGNITIVE COMPUTING, 2023, 7 (03)