From regular expressions to DFA's using compressed NFA's

There are two principal methods for turning regular expressions into NFA's - one due to McNaughton and Yamada and another due to Thompson. Unfortunately, both have drawbacks. Given a regular expression R of length r and with s occurrences of alphabet symbols, Chang and Paige (1992) and Bruggemann-Klein(1993) gave Theta(m+r) time and O(r) spate algorithms to produce a Theta(m) space representation of McNaughton and Yamada's NFA with s+1 states and in transitions. The problem with this NFA is that m=Theta(s(2)) in the worst case. Thompson's method takes Theta(r) time and space to construct a Theta(r) space NFA with Theta(r) states and Theta(r) transitions. The problem with this NFA is that r can be arbitrarily larger than s. We overcome drawbacks of both methods with a Theta(r) time Theta(s) space algorithm to construct an O(s) space representation of McNaughton and Yamada's NFA. Given any set V of NFA states, our representation can be used to compute the set U of states one transition away from the states in V in optimal time O(\V\+\U\). McNaughton and Yamada's NFA requires Theta(\V\x\U\) time in the worst case. Using Thompson's NFA, the equivalent calculation requires Theta(r) time in the worst case. Comparative benchmarks show that an implementation of our method outperforms implementations of competing methods with respect to time for NFA construction, NFA accepting testing, and NFA-to-DFA conversion by subset construction. Throughout this paper program transformations are used to design algorithms and derive programs. A transformation of special importance is a form of finite differencing used previously by Douglas Smith to improve the efficiency of functional programs.