Using words and phonetic strings for efficient information retrieval from imperfectly transcribed spoken documents

Searching for relevant material in documents containing transcription errors presents new challenges for Information Retrieval. This paper examines information retrieval effectiveness on a corpus of spoken broadcast news documents. For documents transcribed using speech recognition, a substantial number of retrieval errors are due to query terms that occur in the spoken document, but are not transcribed because they are not within the speech recognition system's lexicon, even if that lexicon contains twenty thousand words. It has been shown that a phonetic lattice search in conjunction with full word search regains some of the information lost due to out-of-vocabulary words. In this paper an efficient alternative to this search is proposed that does not require a complete search of the phoneme lattices for all documents at run-time. By using fixed length strings of phonemes instead of phonetic lattices, an information retrieval system can search the phoneme space of a spoken document just as efficiently as a normal word document collection. Experimental evidence is presented that this technique permits the system to recapture some of the information lost due to out-of-vocabulary words in the speech recognition transcripts.