Design and Implementation of Vertical Search Engine Based on Hadoop

被引：1
|
作者：
Cheng Lin [1 ]
Ma Yajie [1 ]
机构：
[1] Wuhan Univ Sci & Technol, Coll Informat Sci & Engn, Wuhan 430081, Hubei, Peoples R China
来源：
PROCEEDINGS 2016 EIGHTH INTERNATIONAL CONFERENCE ON MEASURING TECHNOLOGY AND MECHATRONICS AUTOMATION ICMTMA 2016 | 2016年
关键词：
Hadoop; Vertical Search Engine; MapReduce; Efficiency; Precision;
D O I：
10.1109/ICMTMA.2016.58
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
The centralized search engine has problems of excessive server load and limited extended ability when dealing with the massive Internet information. And the search results of general search engine is not so accurate. To solve these problems, a vertical search engine based on Hadoop called HVSE was designed and developed. HVSE was based on the basic principle of the traditional search engine. It improved the current algorithms of topic oriented web crawler, worked in the distributed cluster environment, used the Lucene and other technologies, combined with MapReduce programming model to carry out data processing. Demonstrated by the experiment, the efficiency of HVSE is higher than that of the centralized search engine when dealing with massive data, and the precision of the retrieval results is higher than that of the general search engine.
引用
收藏
页码：199 / 205
页数：7
相关论文
共 50 条
  • [1] The design and implementation of computer vertical search engine
    Wang-wei
    [J]. 2015 SEVENTH INTERNATIONAL CONFERENCE ON MEASURING TECHNOLOGY AND MECHATRONICS AUTOMATION (ICMTMA 2015), 2015, : 1168 - 1171
  • [2] Design and Implementation of One Vertical Video Search Engine
    Liang, Yingyi
    He, Zhenyu
    Li, Yi
    [J]. 2014 INTERNATIONAL CONFERENCE ON SECURITY, PATTERN ANALYSIS, AND CYBERNETICS (SPAC), 2014, : 75 - 79
  • [3] Design and Implementation of Agricultural Information Resources Vertical Search Engine Based on Nutch
    Ding, Enjun
    [J]. 3RD INTERNATIONAL CONFERENCE ON APPLIED ENGINEERING, 2016, 51 : 619 - 624
  • [4] Design and Implementation of Digital Products Vertical Search Engine Based on Android Client
    Huang, Xixin
    Zhou, Lanjiang
    Wang, Chaobin
    [J]. 2013 INTERNATIONAL CONFERENCE ON INFORMATION SCIENCE AND TECHNOLOGY (ICIST), 2013, : 828 - 831
  • [5] Parallelization of Vertical Search Engine using Hadoop and MapReduce
    Pasari, Rajat
    Chaudhari, Vaibhav
    Borkar, Atharva
    Joshi, Amit
    [J]. INTERNATIONAL CONFERENCE ON ADVANCES IN INFORMATION COMMUNICATION TECHNOLOGY & COMPUTING, 2016, 2016,
  • [6] Design of a vertical search engine for synchrotron data: a big data approach using Hadoop ecosystem
    Ali Khaleghi
    Kamran Mahmoudi
    Sonia Mozaffari
    [J]. SN Applied Sciences, 2019, 1
  • [7] Design of a vertical search engine for synchrotron data: a big data approach using Hadoop ecosystem
    Khaleghi, Ali
    Mahmoudi, Kamran
    Mozaffari, Sonia
    [J]. SN APPLIED SCIENCES, 2019, 1 (12)
  • [8] A RESEARCH ON DISTRIBUTED SEARCH ENGINE BASED ON HADOOP
    Rui, Liu
    Lei, Cao
    [J]. 2011 3RD INTERNATIONAL CONFERENCE ON COMPUTER TECHNOLOGY AND DEVELOPMENT (ICCTD 2011), VOL 2, 2012, : 141 - 145
  • [9] Research Distributed Search Engine Based on Hadoop
    Gu, Rui
    [J]. 2015 International Conference on Network and Information Systems for Computers (ICNISC), 2015, : 373 - 375
  • [10] Design and Implementation of Search Engine Based on Big Data
    Zhang Zhifeng
    Han Susu
    [J]. AGRO FOOD INDUSTRY HI-TECH, 2017, 28 (01): : 1355 - 1359
12345