Online Grammar Transformation based on Re-Pair Algorithm

The Re-Pair algorithm (Re-Pair), proposed by Larsson and Moffat, is a simple grammar based compression method that achieves a good compression ratio. Although Re-Pair runs in O(n) time and space for an input of length n, it cannot be used with a large input, because it runs offline and consumes substantial memory space. In this paper, we propose an online grammar transformation algorithm based on a modified Re-Pair along with a compression method using the algorithm. The proposed algorithm runs in O(n log (h) over cap) time using O(g) space, where g and (h) over cap are the number of production rules in a grammar and the maximum height of syntax trees generated by the rules, respectively. We implemented our method and demonstrated that it significantly reduces memory usage with little sacrifice of compression ratio in comparison with the original Re-Pair.