Alphabet Partitioning for Compressed Rank/Select and Applications

We present a data structure that stores a string s[1..n] over the alphabet [1..sigma] in nH(0)(s) o(n)(H-0(s)+1) bits, where H-0(s) is the zero-order entropy of s. This data structure supports the queries access and rank in time O (lg lg sigma), and the select query in constant time. This result improves on previously known data structures using nH(0)(s) + o(n lg sigma) bits, where on highly compressible instances the redundancy o(n lg sigma) cease to be negligible compared to the nH(0)(s) bits that encode the data. The technique is based on combining previous results through an ingenious partitioning of the alphabet, and practical enough to be implementable. It applies not only to strings, but also to several other compact data structures. For example, we achieve (i) faster search times and lower redundancy for the smallest existing full-text self-index; (ii) compressed permutations pi with times for pi() and pi(-1) improved to log-logarithmic; and (iii) the first compressed representation of dynamic collections of disjoint sets.