Uncertainty-aware non-autoregressive neural machine translation

Most existing non-autoregressive neural machine translation (NAT) models generally employ the posterior probability to indicate the model confidence during training, which seems to lag behind the novel uncertainty estimations (UEs) methods successfully deployed in other natural language processing (NLP) tasks. Previous research has practically ignored the large-scale exploration of UE methods in the NAT problem. In this paper, we propose a strategy based on Active Learning employed to investigate whether these sophisticated uncertainty -aware methods are more effective in the NAT problem. Besides, we provide an in-depth analysis of the impact of different widely employed UE methods and propose several tailored ones. In the end, we incorporate these exceptional ones into the practical one-pass GLAT model to obtain enhanced performance. Experimental results demonstrate that sophisticated uncertainty-aware UE methods with the two-step training paradigm are potentially superior to represent the model confidence in facilitating token-level decision-making compared to the posterior probability in NAT to a certain extent.