A Two-Stage Method for Active Learning of Statistical Grammars

Active learning reduces the amount of manually annotated sentences necessary when training state-of-the-art statistical parsers. One popular method, uncertainty sampling, selects sentences for which the parser exhibits low certainty. However, this method does not quantify confidence about the current statistical model itself. In particular, we should be less confident about selection decisions based on low frequency events. We present a novel two-stage method which first targets sentences which cannot be reliably selected using uncertainty sampling, and then applies standard uncertainty sampling to the remaining sentences. An evaluation shows that this method performs better than pure uncertainty sampling, and better than an ensemble method based on bagged ensemble members only.