Text Summary Generation Model Based on Sentence Fusion and Self-Supervised Training

To improve the capability of sentence fusion of deep neural network text generation technique, a text summary generation model based on sentence fusion and self-supervised training is proposed. Before the model training, the training data are firstly pre-processed according to the concept of points of correspondence in the theory of sentence fusion, and thus the data can meet the needs of model training. The training of the proposed model falls into two parts. In the first stage, according to the distribution of the sentence fusion phenomenon in the dataset, the training task of the permutation language model is designed with the points of correspondence as the minimum semantic unit to enhance the ability to capture the information of the fused sentence context. In the second stage, an attention masking strategy based on the fusion information is utilized to control the information intake of the model during the text generation process to enhance the fusion ability in the text generation stage. Experiments on the open dataset show that the proposed model is superior in several evaluation metrics, including those based on statistics, deep semantics and sentence fusion ratio. © 2022, Science Press. All right reserved.