Enhancing multilingual speech recognition in air traffic control by sentence-level language identification

Automatic speech recognition (ASR) technique is becoming increasingly popular to improve the efficiency and safety of air traffic control (ATC) operations. However, the conversation between ATC controllers and pilots using multilingual speech brings a great challenge to building high-accuracy ASR systems. In this work, we present a two-stage multilingual ASR framework. The first stage is to train a language identifier model (LIM), that is based on a recurrent neural network (RNN) to obtain sentence-level language identification (SLID) in the form of one-hot encoding. The second stage aims to train an RNN-based end-to-end multilingual recognition model that utilizes SLID generated by LIM to enhance input features. In this work, we introduce Feature-wise Linear Modulation (FiLM) to improve the performance of multilingual ASR by utilizing SLID. Furthermore, we introduce a new learning module called SLIL, which consists of a FiLM layer and a Squeeze-and-Excitation Networks layer. Extensive experiments on the ATCSpeech dataset show that our proposed method outperforms the baseline model. Compared to the vanilla FiLMed backbone model, the proposed multilingual ASR model obtains about 7.50% character error rate relative performance improvement.