Spark : A Big Data Processing Platform Based On Memory Computing

被引：25
|
作者：
Han, Zhijie [1 ]
Zhang, Yujie [2 ]
机构：
[1] Nanjing Univ Posts & Telecommun, Inst Data & Knowledge Engn, Nanjing, Jiangsu, Peoples R China
[2] Henan Univ, Inst Data & Knowledge Engn, Kaifeng, Henan, Peoples R China
来源：
2015 SEVENTH INTERNATIONAL SYMPOSIUM ON PARALLEL ARCHITECTURES, ALGORITHMS AND PROGRAMMING (PAAP) | 2015年
关键词：
Spark; Memory Computing; Spark SQL; MLlib; GraphX; Spark Streaming;
D O I：
10.1109/PAAP.2015.41
中图分类号：
TP3 [计算技术、计算机技术];
学科分类号：
0812 ;
摘要：
Spark is a memory-based computing framework which has a better ability of computing and fault tolerance, supports batch, interactive, iterative and flow calculations. In this paper, we analyze the Spark's primary framework, core technologies, and point out the advantages and disadvantages of the Spark. In the end, we make a discussion for the future trends of the Spark technologies.
引用
收藏
页码：172 / 176
页数：5
相关论文
共 50 条
  • [1] A Spark-Based Big Data Platform for Massive Remote Sensing Data Processing
    Sun, Zhongyi
    Chen, Fengke
    Chi, Mingmin
    Zhu, Yangyong
    [J]. DATA SCIENCE, 2015, 9208 : 120 - 126
  • [2] SPARK-A Big Data Processing Platform for Machine Learning
    Fu, Jian
    Sun, Junwei
    Wang, Kaiyuan
    [J]. 2016 2ND INTERNATIONAL CONFERENCE ON INDUSTRIAL INFORMATICS - COMPUTING TECHNOLOGY, INTELLIGENT TECHNOLOGY, INDUSTRIAL INFORMATION INTEGRATION (ICIICII), 2016, : 48 - 51
  • [3] A Prototype of Healthcare Big Data Processing System Based on Spark
    Liu, Wenzhi
    Li, Qi
    Cai, Yunpeng
    Li, Ye
    Li, Xiaoyan
    [J]. 2015 8TH INTERNATIONAL CONFERENCE ON BIOMEDICAL ENGINEERING AND INFORMATICS (BMEI), 2015, : 516 - 520
  • [4] Review of big data processing based on granular computing
    Xu, Ji
    Wang, Guo-Yin
    Yu, Hong
    [J]. Jisuanji Xuebao/Chinese Journal of Computers, 2015, 38 (08): : 1497 - 1517
  • [5] A Design of Processing Platform for Smartphone Based on Big Data
    Zhang, Jie
    Zhang, Ke
    Zhou, Hengxin
    [J]. 2015 8TH INTERNATIONAL SYMPOSIUM ON COMPUTATIONAL INTELLIGENCE AND DESIGN (ISCID), VOL 1, 2015, : 162 - 165
  • [6] Architecture Design of Distributed Medical Big Data Platform Based on Spark
    Tu, Yongqiu
    Lu, Yiqiang
    Chen, Guohua
    Zhao, Jie
    Yi, Faling
    [J]. PROCEEDINGS OF 2019 IEEE 8TH JOINT INTERNATIONAL INFORMATION TECHNOLOGY AND ARTIFICIAL INTELLIGENCE CONFERENCE (ITAIC 2019), 2019, : 682 - 685
  • [7] Apriori algorithm optimization based on Spark platform under big data
    Yu, Huafeng
    [J]. MICROPROCESSORS AND MICROSYSTEMS, 2021, 80
  • [8] The Research of Large Scale Data Processing Platform Based on the Spark
    Na, Chu
    Xin, Cao
    [J]. 2016 INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION, BIG DATA & SMART CITY (ICITBS), 2017, : 293 - 296
  • [9] Big Data Platform for Oil and Gas Production Based on Apache Spark
    Qing, Peng
    Li, Yi
    Luo, Shuqin
    Xu, Zhuoqun
    [J]. MODERN INDUSTRIAL IOT, BIG DATA AND SUPPLY CHAIN, IIOTBDSC 2020, 2021, 218 : 129 - 141
  • [10] Research and Application of Spark Platform on Big Data Processing in Intelligent Agriculture of Jilin Province
    Fu, Siwei
    Chen, Guifen
    Zhao, Shan
    Xiao, Enze
    [J]. COMPUTER AND COMPUTING TECHNOLOGIES IN AGRICULTURE XI, CCTA 2017, PT II, 2019, 546 : 1 - 12
12345