Shape Recognition Based on MapReduce and In-Memory Processing on Distributed File System

被引：0

作者：

Baik, Namkyun ^{[1
]}

Hazra, Dipankar ^{[2
]}

Bhattacharyya, Debnath ^{[3
]}

机构：

[1] Korea Adv Agcy Convergence Technol, IT Venture Tower,135 Jungdae Ro, Seoul, South Korea

[2] OmDayal Grp Inst, Comp Sci & Engn Dept, Howrah 711316, W Bengal, India

[3] Vignans Inst Informat Technol, Comp Sci & Engn Dept, Visakhapatnam 530049, Andhra Pradesh, India

来源：

INTERNATIONAL JOURNAL OF GRID AND DISTRIBUTED COMPUTING | 2018年 / 11卷 / 02期

关键词：

Shape Retrieval; Distributed File System; HDFS; MapReduce; In-Memory Processing; Spark; Image Big Data;

D O I：

10.14257/ijgdc.2018.11.2.03

中图分类号：

TP31 [计算机软件];

学科分类号：

081202 ; 0835 ;

摘要：

Two novel approaches for centroid-radii based shape retrieval on distributed file system are proposed in this paper. Modified Centroid-Radii model is used for calculating the shape features of trained images. These shape features are stored into Hadoop Distributed File System (HDFS) instead of relational database, generally used for feature storage. HDFS can store large number of shapes that is not possible to be stored in a single machine. Modified Centroid-Radii Model is also used to calculate the shape feature of query image. In one approach MapReduce query is used for recognizing binary shape. In another approach Apache Spark is used. Shape feature of query shape is compared with the shape features stored in HDFS. In-memory processing of Apache Spark used to increase the speed of retrieval process. Spark based image retrieval is faster than MapReduce based image retrieval.

引用

页码：21 / 30

页数：10

共 50 条

[41] POSTER: MemFS: an In-Memory Runtime File System with Symmetrical Data Distribution
Uta, Alexandra
Sandu, Andreea
Kielmann, Thilo
[J]. 2014 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING (CLUSTER), 2014, : 272 - 273
[42] The Design and Implementation of an Efficient User-Space In-memory File System
Sha, Edwin H. -M.
Jia, Yang
Chen, Xianzhang
Zhuge, Qingfeng
Jiang, Weiwen
Qin, Jiejie
[J]. 2016 5TH NON-VOLATILE MEMORY SYSTEMS AND APPLICATIONS SYMPOSIUM (NVMSA), 2016,
[43] Implementation of Distributed In-Memory Moving Objects Management System
Lee, H.
Kwak, Y.
Song, S.
[J]. ADVANCED SCIENCE LETTERS, 2017, 23 (10) : 10361 - 10365
[44] Optimizing Pipelined Execution for Distributed In-Memory OLAP System
Wang, Li
Zhang, Lei
Yu, Chengcheng
Zhou, Aoying
[J]. DATABASE SYSTEMS FOR ADVANCED APPLICATIONS, DASFAA 2014, 2014, 8505 : 204 - 216
[45] LUPIS: Latch-Up Based Ultra Efficient Processing In-Memory System
Sim, Joonseop
Imani, Mohsen
Choi, Woojin
Kim, Yeseong
Rosing, Tajana
[J]. 2018 19TH INTERNATIONAL SYMPOSIUM ON QUALITY ELECTRONIC DESIGN (ISQED), 2018, : 55 - 60
[46] DISE: A Distributed in-Memory SPARQL Processing Engine over Tensor Data
Jabeen, Hajira
Haziiev, Eskender
Sejdiu, Gezim
Lehmann, Jens
[J]. 2020 IEEE 14TH INTERNATIONAL CONFERENCE ON SEMANTIC COMPUTING (ICSC 2020), 2020, : 400 - 407
[47] A dynamic migration algorithm for a distributed memory-based file management system
Griffioen, J
Anderson, TA
Breitbart, Y
[J]. SEVENTH INTERNATIONAL WORKSHOP ON RESEARCH ISSUES IN DATA ENGINEERING, PROCEEDINGS: HIGH PERFORMANCE DATABASE MANAGEMENT FOR LARGE-SCALE APPLICATIONS, 1997, : 151 - 160
[48] SilverChunk: An Efficient In-Memory Parallel Graph Processing System
Zheng, Tianqi
Zhang, Zhibin
Cheng, Xueqi
[J]. DATABASE AND EXPERT SYSTEMS APPLICATIONS, PT II, 2019, 11707 : 222 - 236
[49] Design and Implementation of a Novel Distributed Memory File System
Karnani, Urvashi
Kalmady, Rajesh
Chand, Phool
Bhattacharjee, Anup
Jagadeesh, B. S.
[J]. ADVANCED COMPUTING, PT III, 2011, 133 : 139 - +
[50] A Consistency Mechanism for Distributed Persistent Memory File System
Chen B.
Lu Y.
Cai T.
Chen Y.
Tu Y.
Shu J.
[J]. Jisuanji Yanjiu yu Fazhan/Computer Research and Development, 2020, 57 (03): : 660 - 667

← 1 2 3 4 5 →