Shape Recognition Based on MapReduce and In-Memory Processing on Distributed File System

被引：0

作者：

Baik, Namkyun ^{[1
]}

Hazra, Dipankar ^{[2
]}

Bhattacharyya, Debnath ^{[3
]}

机构：

[1] Korea Adv Agcy Convergence Technol, IT Venture Tower,135 Jungdae Ro, Seoul, South Korea

[2] OmDayal Grp Inst, Comp Sci & Engn Dept, Howrah 711316, W Bengal, India

[3] Vignans Inst Informat Technol, Comp Sci & Engn Dept, Visakhapatnam 530049, Andhra Pradesh, India

来源：

INTERNATIONAL JOURNAL OF GRID AND DISTRIBUTED COMPUTING | 2018年 / 11卷 / 02期

关键词：

Shape Retrieval; Distributed File System; HDFS; MapReduce; In-Memory Processing; Spark; Image Big Data;

D O I：

10.14257/ijgdc.2018.11.2.03

中图分类号：

TP31 [计算机软件];

学科分类号：

081202 ; 0835 ;

摘要：

Two novel approaches for centroid-radii based shape retrieval on distributed file system are proposed in this paper. Modified Centroid-Radii model is used for calculating the shape features of trained images. These shape features are stored into Hadoop Distributed File System (HDFS) instead of relational database, generally used for feature storage. HDFS can store large number of shapes that is not possible to be stored in a single machine. Modified Centroid-Radii Model is also used to calculate the shape feature of query image. In one approach MapReduce query is used for recognizing binary shape. In another approach Apache Spark is used. Shape feature of query shape is compared with the shape features stored in HDFS. In-memory processing of Apache Spark used to increase the speed of retrieval process. Spark based image retrieval is faster than MapReduce based image retrieval.

引用

页码：21 / 30

页数：10

共 50 条

[1] Performance Optimization of In-Memory File System in Distributed Storage System
Li, Zhaowei
Yan, Yunlong
Mo, Jintao
Wen, Zhaocong
Wu, Junmin
[J]. 2017 INTERNATIONAL CONFERENCE ON NETWORKING, ARCHITECTURE, AND STORAGE (NAS), 2017, : 280 - 281
[2] IMFSSC: An In-Memory Distributed File System Framework for Super Computing
Li, Binyang
Li, Bo
Liu, Ming
[J]. 2016 7TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND BIG DATA (CCBD), 2016, : 132 - 137
[3] Enhancing in-memory efficiency for MapReduce-based data processing
Veiga, Jorge
Exposito, Roberto R.
Taboada, Guillermo L.
Tourino, Juan
[J]. JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING, 2018, 120 : 323 - 338
[4] Dima: A Distributed In-Memory Similarity-Based Query Processing System
Sun, Ji
Shang, Zeyuan
Li, Guoliang
Deng, Dong
Bao, Zhifeng
[J]. PROCEEDINGS OF THE VLDB ENDOWMENT, 2017, 10 (12): : 1925 - 1928
[5] DISTIL: A Distributed In-Memory Data Processing System for Location-Based Services
Patrou, Maria
Alam, Md Mahbub
Memarzia, Puya
Ray, Suprio
Bhavsar, Virendra C.
Kent, Kenneth B.
Dueck, Gerhard W.
[J]. 26TH ACM SIGSPATIAL INTERNATIONAL CONFERENCE ON ADVANCES IN GEOGRAPHIC INFORMATION SYSTEMS (ACM SIGSPATIAL GIS 2018), 2018, : 496 - 499
[6] A New Design of In-Memory File System Based on File Virtual Address Framework
Sha, Edwin H. -M.
Chen, Xianzhang
Zhuge, Qingfeng
Shi, Liang
Jiang, Weiwen
[J]. IEEE TRANSACTIONS ON COMPUTERS, 2016, 65 (10) : 2959 - 2972
[7] PHOENIX - A SAFE IN-MEMORY FILE SYSTEM
GAIT, J
[J]. COMMUNICATIONS OF THE ACM, 1990, 33 (01) : 81 - 86
[8] MemEFS: A network-aware elastic in-memory runtime distributed file system
Uta, Alexandru
Danner, Ove
van der Weegen, Cas
Oprescu, Ana-Maria
Sandu, Andreea
Costache, Stefania
Kielmann, Thilo
[J]. FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2018, 82 : 631 - 646
[9] MapReuse : Reusing Computation in an In-Memory MapReduce System
Tiwari, Devesh
Solihin, Yan
[J]. 2014 IEEE 28TH INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM, 2014,
[10] Cruncher: Distributed In-Memory Processing for Location-Based Services
Abdelhamid, Ahmed S.
Tang, Mingjie
Aly, Ahmed M.
Mahmood, Ahmed R.
Qadah, Thamir
Aref, Walid G.
Basalamah, Saleh
[J]. 2016 32ND IEEE INTERNATIONAL CONFERENCE ON DATA ENGINEERING (ICDE), 2016, : 1406 - 1409

← 1 2 3 4 5 →