Phrase-based translation of speech recognizer word lattices using loglinear model combination

This paper presents a phrase-based speech translation system that combines phrasal lexicon, language, and acoustic model features in a loglinear model. Automatic speech recognition and machine translation are coupled by using large word lattices as the input for translation. For the first time, all features are directly integrated into the decoding process. The feature weights are iteratively optimized for an objective error measure. We prove that acoustic recognition scores of the recognized words in the lattices together with a source language model score positively and significantly affect the translation quality. We show the advantage of using loglinear model combination for a robust optimization of scaling factors. We report consistent improvements compared with translations of single best recognition output on an Italian-to-English translation task. First encouraging results were also obtained on a large vocabulary task of translating European parliamentary speeches.