Research Distributed Search Engine Based on Hadoop

被引：0
|
作者：
Gu, Rui [1 ]
机构：
[1] Suzhou Ind Pk Inst Serv Outsourcing, Suzhou, Peoples R China
来源：
2015 International Conference on Network and Information Systems for Computers (ICNISC) | 2015年
关键词：
component; Hadoop; HBase; MapReduce; index file; Lucene;
D O I：
10.1109/ICNISC.2015.149
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
In the age internet, processing massive data appears bottlenecks based on the Lucene. In order to improve the timeliness of the massive data retrieval, this paper focuses on the research of distributed search engine based on Hadoop. First, this paper introduces the Hadoop Distributed File System (HDFS), MapReduce parallel programming model, HBase database. Then through the MapReduce calculation model build index file and store it to cluster HBase. At last, the final experiment shows the advantages of distributed search engine based on Hadoop.
引用
收藏
页码：373 / 375
页数：3
相关论文
共 50 条
  • [1] A RESEARCH ON DISTRIBUTED SEARCH ENGINE BASED ON HADOOP
    Rui, Liu
    Lei, Cao
    [J]. 2011 3RD INTERNATIONAL CONFERENCE ON COMPUTER TECHNOLOGY AND DEVELOPMENT (ICCTD 2011), VOL 2, 2012, : 141 - 145
  • [2] Investigation on Hadoop-based distributed search engine
    Chen, Ning
    Xiangyang, Chai
    [J]. Journal of Software Engineering, 2014, 8 (03): : 127 - 131
  • [3] Distributed Content Based Image Search Engine using Hadoop Framework
    Uttarwar, Dhananjay
    Agarwal, Aakash
    Kadiwar, Riyaz
    Katkar, Vijay D.
    [J]. 2017 INTERNATIONAL CONFERENCE ON COMMUNICATION AND SIGNAL PROCESSING (ICCSP), 2017, : 1706 - 1710
  • [4] Design and Implementation of Vertical Search Engine Based on Hadoop
    Cheng Lin
    Ma Yajie
    [J]. PROCEEDINGS 2016 EIGHTH INTERNATIONAL CONFERENCE ON MEASURING TECHNOLOGY AND MECHATRONICS AUTOMATION ICMTMA 2016, 2016, : 199 - 205
  • [5] Hydra: a scalable proteomic search engine which utilizes the Hadoop distributed computing framework
    Steven Lewis
    Attila Csordas
    Sarah Killcoyne
    Henning Hermjakob
    Michael R Hoopmann
    Robert L Moritz
    Eric W Deutsch
    John Boyle
    [J]. BMC Bioinformatics, 13
  • [6] Hydra: a scalable proteomic search engine which utilizes the Hadoop distributed computing framework
    Lewis, Steven
    Csordas, Attila
    Killcoyne, Sarah
    Hermjakob, Henning
    Hoopmann, Michael R.
    Moritz, Robert L.
    Deutsch, Eric W.
    Boyle, John
    [J]. BMC BIOINFORMATICS, 2012, 13
  • [7] Research and Practice of Distributed Parallel Search Algorithm on Hadoop_MapReduce
    Duan, AiLing
    Si, HaiFang
    [J]. 2012 INTERNATIONAL CONFERENCE ON CONTROL ENGINEERING AND COMMUNICATION TECHNOLOGY (ICCECT 2012), 2012, : 105 - 108
  • [8] Geospatial Hadoop (GS-Hadoop) An efficient MapReduce based engine for distributed processing of Shapefiles
    Abdul, Jhummarwala
    Alkathiri, Mazin
    Potdar, M. B.
    [J]. 2016 2ND INTERNATIONAL CONFERENCE ON ADVANCES IN COMPUTING, COMMUNICATION, & AUTOMATION (ICACCA) (FALL), 2016, : 22 - 28
  • [9] Research and Realization of News Gathering and Editing System Based on Distributed Search Engine
    Han, Yamin
    Liu, Kun
    Ma, Kun
    [J]. ADVANCES IN INTELLIGENT NETWORKING AND COLLABORATIVE SYSTEMS, INCOS-2017, 2018, 8 : 349 - 354
  • [10] Analysis of distributed computing architecture search principle based on Hadoop
    Duan, Ailing
    Cao, Dan
    Si, Haifang
    [J]. COMPUTER AND INFORMATION TECHNOLOGY, 2014, 519-520 : 54 - 57
12345