CSIndex: A Coprocessor-Based Classified Secondary Index Mechanism for Efficient HBase Query

In the era of big data, HBase has been widely used in many business areas due to its good performance of massive data storage and management. Unfortunately, the native HBase only optimizes index for rowkey without creating indexes to the non-key column. A full table scan has to be used when querying the non-key column data, which greatly affects the efficiency of complex condition queries. In this paper, we present CSIndex, which is a coprocessor-based classified secondary index mechanism in HBase. CSIndex proposes the Observer-based secondary index management model to ensure the colocation of relevant data. Furthermore, according to the different data characteristics and query requirements, CSIndex designs the classified memory index model to balance query performance and index performance. On this basis, CSIndex proposes the Endpoint-based parallel query algorithm to reduce data transmission overhead, which improves query performance effectively. Finally, experiments are conducted on real datasets of vehicle trajectory. The results show that the query performance of CSIndex is significantly improved compared with the Solr-based scheme and HiBase, and has better overall performance.