Effective Multi-stream Joining in Apache Samza Framework

被引：10

作者：

Zhuang, Zhenyun ^{[1
]}

Feng, Tao ^{[1
]}

Pan, Yi ^{[1
]}

Ramachandra, Haricharan ^{[1
]}

Sridharan, Badri ^{[1
]}

机构：

[1] LinkedIn Corp, 2029 Stierlin Court, Mountain View, CA 94043 USA

来源：

2016 IEEE INTERNATIONAL CONGRESS ON BIG DATA - BIGDATA CONGRESS 2016 | 2016年

关键词：

Multi-stream joining; Samza; Stream processing; Big Data;

D O I：

10.1109/BigDataCongress.2016.41

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Increasing adoption of Big Data in business environments have driven the needs of stream joining in realtime fashion. Multi-stream joining is an important stream processing type in todays Internet companies, and it has been used to generate higher-quality data in business pipelines. Multi-stream joining can be performed in two models: ( 1) All-In-One (AIO) Joining and (2) Step-By-Step (SBS) Joining. Both models have advantages and disadvantages with regard to memory footprint, joining latency, deployment complexity, etc. In this work, we analyze the performance tradeoffs associated with these two models using Apache Samza.

引用

页码：267 / 274

页数：8

共 50 条

[1] Employing heterogeneous information in a multi-stream framework
Christensen, H
Lindberg, B
Andersen, O
[J]. 2000 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, PROCEEDINGS, VOLS I-VI, 2000, : 1571 - 1574
[2] Incorporating phonetic knowledge into a multi-stream HMM framework
Norouzian, Atta
Selouani, Sid-Ahmed
O'Shaughnessy, Douglas
[J]. 2008 CANADIAN CONFERENCE ON ELECTRICAL AND COMPUTER ENGINEERING, VOLS 1-4, 2008, : 1632 - +
[3] Multi-stream inflation
Li, Miao
Wang, Yi
[J]. JOURNAL OF COSMOLOGY AND ASTROPARTICLE PHYSICS, 2009, (07):
[4] Multi-stream MPA
Bestler, Caitlin
[J]. 2005 IEEE International Conference on Cluster Computing (CLUSTER), 2006, : 623 - 628
[5] Multi-stream Deep Learning Framework for Automated Presentation Assessment
Li, Junnan
Wong, Yongkang
Kankanhalli, Mohan S.
[J]. PROCEEDINGS OF 2016 IEEE INTERNATIONAL SYMPOSIUM ON MULTIMEDIA (ISM), 2016, : 222 - 225
[6] A Multi-Stream Sequence Learning Framework for Human Interaction Recognition
Haroon, Umair
Ullah, Amin
Hussain, Tanveer
Ullah, Waseem
Sajjad, Muhammad
Muhammad, Khan
Lee, Mi Young
Baik, Sung Wook
[J]. IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, 2022, 52 (03) : 435 - 444
[7] A WFST Framework for Single-Pass Multi-Stream Decoding
Xu, Sirui
Fosler-Lussier, Eric
[J]. 17TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2016), VOLS 1-5: UNDERSTANDING SPEECH PROCESSING IN HUMANS AND MACHINES, 2016, : 1908 - 1912
[8] A flexible and robust modelling framework for multi-stream heat exchangers
Skaugen, Geir
Kolsaker, Kjell
Walnum, Harald Taxt
Wilhelmsen, Oivind
[J]. COMPUTERS & CHEMICAL ENGINEERING, 2013, 49 : 95 - 104
[9] A MULTI-STREAM ASR FRAMEWORK FOR BLSTM MODELING OF CONVERSATIONAL SPEECH
Woellmer, Martin
Eyben, Florian
Schuller, Bjoern
Rigoll, Gerhard
[J]. 2011 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 2011, : 4860 - 4863
[10] Stream weight computation for multi-stream classifiers
Potamianos, Alexandros
Sanchez-Soto, Eduardo
Daoudi, Khalid
[J]. 2006 IEEE International Conference on Acoustics, Speech and Signal Processing, Vols 1-13, 2006, : 353 - 356

← 1 2 3 4 5 →