Can LSH (locality-sensitive hashing) be replaced by neural network?

被引:0
|
作者
Renyang Liu
Jun Zhao
Xing Chu
Yu Liang
Wei Zhou
Jing He
机构
[1] Yunnan University,School of Information Science and Engineering
[2] Yunnan University,School of Software
[3] Yunnan University,Engineering Research Center of Cyberspace
[4] Didi Chuxing,undefined
来源
Soft Computing | 2024年 / 28卷
关键词
Learned index; Deep learning; Locality-sensitive hashing; KNN;
D O I
暂无
中图分类号
学科分类号
摘要
With the rapid development of GPU (graphics processing unit) technologies and neural networks, we can explore more appropriate data structures and algorithms. Recent progress shows that neural networks can partly replace traditional data structures. In this paper, we proposed a novel DNN (deep neural network)-based learned locality-sensitive hashing, called LLSH, to efficiently and flexibly map high-dimensional data to low-dimensional space. LLSH replaces the traditional LSH (locality-sensitive hashing) function families with parallel multi-layer neural networks, which reduces the time and memory consumption and guarantees query accuracy simultaneously. The proposed LLSH demonstrate the feasibility of replacing the hash index with learning-based neural networks and open a new door for developers to design and configure data organization more accurately to improve information-searching performance. Extensive experiments on different types of data sets show the superiority of the proposed method in query accuracy, time consumption, and memory usage.
引用
收藏
页码:1041 / 1053
页数:12
相关论文
共 50 条
  • [1] Can LSH (locality-sensitive hashing) be replaced by neural network?
    Liu, Renyang
    Zhao, Jun
    Chu, Xing
    Liang, Yu
    Zhou, Wei
    He, Jing
    [J]. SOFT COMPUTING, 2024, 28 (02) : 887 - 902
  • [2] BCH-LSH: a new scheme of locality-sensitive hashing
    Ma, Yuena
    Feng, Xiaoyi
    Liu, Yang
    Li, Shuhong
    [J]. IET IMAGE PROCESSING, 2018, 12 (06) : 850 - 855
  • [3] In Defense of Locality-Sensitive Hashing
    Ding, Kun
    Huo, Chunlei
    Fan, Bin
    Xiang, Shiming
    Pan, Chunhong
    [J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2018, 29 (01) : 87 - 103
  • [4] Kernelized Locality-Sensitive Hashing
    Kulis, Brian
    Grauman, Kristen
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2012, 34 (06) : 1092 - 1104
  • [5] Correlated Locality-Sensitive Hashing
    Pagh, Rasmus
    [J]. ALGORITHMS - ESA 2015, 2015, 9294
  • [6] DB-LSH: Locality-Sensitive Hashing with Query-based Dynamic Bucketing
    Tian, Yao
    Zhao, Xi
    Thou, Xiaofang
    [J]. 2022 IEEE 38TH INTERNATIONAL CONFERENCE ON DATA ENGINEERING (ICDE 2022), 2022, : 2250 - 2262
  • [7] Locality-Sensitive Hashing for Long Context Neural Machine Translation
    Petrick, Frithjof
    Rosendahl, Jan
    Herold, Christian
    Ney, Hermann
    [J]. PROCEEDINGS OF THE 19TH INTERNATIONAL CONFERENCE ON SPOKEN LANGUAGE TRANSLATION (IWSLT 2022), 2022, : 32 - 42
  • [8] An Improved Algorithm for Locality-Sensitive Hashing
    Cen, Wei
    Miao, Kehua
    [J]. 10TH INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE & EDUCATION (ICCSE 2015), 2015, : 61 - 64
  • [9] Bit Reduction for Locality-Sensitive Hashing
    Liu, Huawen
    Zhou, Wenhua
    Zhang, Hong
    Li, Gang
    Zhang, Shichao
    Li, Xuelong
    [J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2024, 35 (09) : 12470 - 12481
  • [10] Cross-media retrieval based on locality-sensitive hashing and neural network algorithms
    [J]. 2018, National University of Defense Technology (40):