Evolution of Spark Framework for simplifying Big Data Analytics

被引:0
|
作者
Kumar, Subhash [1 ]
机构
[1] Mumbai Uinv, St Xaviers Coll, Bombay, Maharashtra, India
关键词
Big Data; ecosystem; flink; petabytes; scala; spark;
D O I
暂无
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Most of the data was generated (about 80%) in last 2 years. Such data is captured from different sources at high velocity and comes from variety of sources. When data size reaches petabytes or more, current desktop and other system cannot handle it and requires efficient management of storage and retrieval. Such data is called Big Data. This Big Data is managed by an efficient framework called Spark. Big Data Analytics requires quick response. Spark addresses it. This paper discusses how framework Spark can be used for Big Data Analytics. Also cpu intensive task can be quickly solved in spark. An illustration of it is given by solving mathematical problem. It focuses on Spark ecosystem. This paper also compares spark with flink framework. Scala, python and java are used in spark. Scala being functional programming language is preferred.
引用
收藏
页码:3597 / 3602
页数:6
相关论文
共 50 条
  • [1] Big data analytics on Apache Spark
    Salloum S.
    Dautov R.
    Chen X.
    Peng P.X.
    Huang J.Z.
    [J]. International Journal of Data Science and Analytics, 2016, 1 (3-4) : 145 - 164
  • [2] Simplifying Big Data Analytics Systems with a Reference Architecture
    Sang, Go Muan
    Xu, Lai
    de Vrieze, Paul
    [J]. COLLABORATION IN A DATA-RICH WORLD, 2017, 506 : 242 - 249
  • [3] Geospatial Big Data Analytics Engine for Spark
    Wang, Shaohua
    Zhong, Yang
    Lu, Hao
    Wang, Erqi
    Yun, Weiying
    Cai, Wenwen
    [J]. BIGSPATIAL 2017: PROCEEDINGS OF THE 6TH ACM SIGSPATIAL INTERNATIONAL WORKSHOP ON ANALYTICS FOR BIG GEOSPATIAL DATA (BIGSPATIAL-2017), 2017, : 42 - 45
  • [4] Debugging Big Data Analytics in Spark with BigDebug
    Gulzar, Muhammad Ali
    Interlandi, Matteo
    Condie, Tyson
    Kim, Miryung
    [J]. SIGMOD'17: PROCEEDINGS OF THE 2017 ACM INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA, 2017, : 1627 - 1630
  • [5] Big Data Software Analytics with Apache Spark
    Gousios, Georgios
    [J]. PROCEEDINGS 2018 IEEE/ACM 40TH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING - COMPANION (ICSE-COMPANION, 2018, : 542 - 543
  • [6] Big Data Analytics with Datalog Queries on Spark
    Shkapsky, Alexander
    Yang, Mohan
    Interlandi, Matteo
    Chiu, Hsuan
    Condie, Tyson
    Zaniolo, Carlo
    [J]. SIGMOD'16: PROCEEDINGS OF THE 2016 INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA, 2016, : 1135 - 1149
  • [7] IRIS: A Goal-Oriented Big Data Analytics Framework on Spark for Better Business Decisions
    Park, Grace
    Park, Sooyong
    Khan, Latifur
    Chung, Lawrence
    [J]. 2017 IEEE INTERNATIONAL CONFERENCE ON BIG DATA AND SMART COMPUTING (BIGCOMP), 2017, : 76 - 83
  • [8] A big data analytics framework for scientific data management
    Fiore, Sandro
    Palazzo, Cosimo
    D'Anca, Alessandro
    Foster, Ian
    Williams, Dean N.
    Aloisio, Giovanni
    [J]. 2013 IEEE INTERNATIONAL CONFERENCE ON BIG DATA, 2013,
  • [9] Decision Support Framework for Big Data Analytics
    Agarwal, Sakshi
    Narayanan, Krishnaprasad
    Sinha, Manjira
    Gupta, Rohit
    Eswaran, Sharanya
    Mukherjee, Tridib
    [J]. 2018 IEEE WORLD CONGRESS ON SERVICES (IEEE SERVICES 2018), 2018, : 53 - 54
  • [10] The framework of talent analytics using big data
    Saputra, Arnold
    Wang, Gunawan
    Zhang, Justin Zuopeng
    Behl, Abhishek
    [J]. TQM JOURNAL, 2022, 34 (01): : 178 - 198