Design and Implementation of Vertical Search Engine Based on Hadoop

被引：1
|
作者：
Cheng Lin [1 ]
Ma Yajie [1 ]
机构：
[1] Wuhan Univ Sci & Technol, Coll Informat Sci & Engn, Wuhan 430081, Hubei, Peoples R China
来源：
PROCEEDINGS 2016 EIGHTH INTERNATIONAL CONFERENCE ON MEASURING TECHNOLOGY AND MECHATRONICS AUTOMATION ICMTMA 2016 | 2016年
关键词：
Hadoop; Vertical Search Engine; MapReduce; Efficiency; Precision;
D O I：
10.1109/ICMTMA.2016.58
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
The centralized search engine has problems of excessive server load and limited extended ability when dealing with the massive Internet information. And the search results of general search engine is not so accurate. To solve these problems, a vertical search engine based on Hadoop called HVSE was designed and developed. HVSE was based on the basic principle of the traditional search engine. It improved the current algorithms of topic oriented web crawler, worked in the distributed cluster environment, used the Lucene and other technologies, combined with MapReduce programming model to carry out data processing. Demonstrated by the experiment, the efficiency of HVSE is higher than that of the centralized search engine when dealing with massive data, and the precision of the retrieval results is higher than that of the general search engine.
引用
收藏
页码：199 / 205
页数：7
相关论文
共 50 条
  • [41] Design and implementation of high-performance FTP search engine
    Guo, Li-Li
    Zhao, Chun-Jiang
    [J]. Huanan Ligong Daxue Xuebao/Journal of South China University of Technology (Natural Science), 2009, 37 (01): : 135 - 139
  • [42] The design and implementation of computer full-text search engine
    Bu Zhi-jing
    Fan Yan
    Yang Jian-wen
    Cheng Lin
    [J]. 2015 SEVENTH INTERNATIONAL CONFERENCE ON MEASURING TECHNOLOGY AND MECHATRONICS AUTOMATION (ICMTMA 2015), 2015, : 1163 - 1167
  • [43] Design and Implementation of a Threaded Search Engine for Tour Recommendation Systems
    Lee, Junghoon
    Park, Gyung-Leen
    Ko, Jin-hee
    Shin, In-Hye
    Kang, Mikyung
    [J]. U- AND E-SERVICE, SCIENCE AND TECHNOLOGY, 2010, 124 : 1 - +
  • [44] Automatic classification of meta-search engine design and implementation
    Cao, Jiandong
    Tang, Yang
    Song, Jian
    [J]. 2ND IEEE INTERNATIONAL CONFERENCE ON ADVANCED COMPUTER CONTROL (ICACC 2010), VOL. 2, 2010, : 425 - 430
  • [45] Design and Implementation of a Scalable Distributed Web Crawler Based on Hadoop
    Shi, YuLiang
    Zhang, Ti
    [J]. 2017 IEEE 2ND INTERNATIONAL CONFERENCE ON BIG DATA ANALYSIS (ICBDA), 2017, : 537 - 541
  • [46] The design and implementation of image parallel processing framework based on Hadoop
    Wang, Shenkuo
    Wu, Shaofei
    Zhang, Huajie
    Xia, Ning
    [J]. BASIC & CLINICAL PHARMACOLOGY & TOXICOLOGY, 2020, 126 : 183 - 183
  • [47] Design and Implementation of Digital Library Retrieval System Based on Hadoop
    Wang, Daying
    [J]. PROCEEDINGS OF THE 2017 3RD INTERNATIONAL CONFERENCE ON SOCIAL SCIENCE AND HIGHER EDUCATION, 2017, 99 : 299 - 302
  • [48] User recommendation implementation in the search engine based on Ajax
    He, Youquan
    Xu, Xiaole
    Tang, Huajiao
    Xu, Cheng
    [J]. Proceedings - 2011 8th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2011, 2011, 3 : 1851 - 1854
  • [49] Design and implementation of Vertical Search Platform for Electronic Product Information
    Wang, Aihua
    [J]. 2017 INTERNATIONAL CONFERENCE ON ROBOTS & INTELLIGENT SYSTEM (ICRIS), 2017, : 101 - 104
  • [50] Design and Implementation of a Topic-Focused Search Engine based on Multi-Agent System
    Zhai, Dongsheng
    Yang, Yang
    [J]. IEEE/SOLI'2008: PROCEEDINGS OF 2008 IEEE INTERNATIONAL CONFERENCE ON SERVICE OPERATIONS AND LOGISTICS, AND INFORMATICS, VOLS 1 AND 2, 2008, : 1035 - 1039
12345