Abstractive Summarization Based on Fine-Grained Interpretable Matrix

被引:0
|
作者
Wang H. [1 ]
Gao Y. [1 ,3 ]
Feng J. [2 ]
Hu M. [2 ]
Wang H. [1 ]
Bai Y. [1 ]
机构
[1] School of Computer Science and Technology, Beijing Institute of Technology, Beijing
[2] China Mobile Research Institute, Beijing
[3] Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, Beijing
关键词
Abstractive summarization; Centrality; Controllable; Interpretable extraction; Mask matrix;
D O I
10.13209/j.0479-8023.2020.082
中图分类号
学科分类号
摘要
According to the great challenge of summarizing and interpreting the information of a long article in the summary model. A summary model (Fine-Grained Interpretable Matrix, FGIM), which is retracted and then generated, is proposed to improve the interpretability of the long text on the significance, update and relevance, and then guide to automatically generate a summary. The model uses a pair-wise extractor to compress the content of the article, capture the sentence with a high degree of centrality, and uses the compressed text to combine with the generator to achieve the process of generating the summary. At the same time, the interpretable mask matrix can be used to control the direction of digest generation at the generation end. The encoder uses two methods based on Transformer and BERT respectively. This method is better than the best baseline model on the benchmark text summary data set (CNN/DailyMail and NYT50). The experiment further builds two test data sets to verify the update and relevance of the abstract, and the proposed model achieves corresponding improvements in the controllable generation of the data set. © 2021 Peking University.
引用
收藏
页码:23 / 30
页数:7
相关论文
共 20 条
  • [1] See A, Liu P J, Manning C D., Get to the point: Summarization with pointer-generator networks, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 1073-1083, (2017)
  • [2] Peters M E, Neumann M, Iyyer M, Et al., Deep contex-tualized word representations
  • [3] Radford A, Narasimhan K, Salimans T, Et al., Im-proving language understanding by generative pre-training
  • [4] Devlin J, Chang M W, Lee K, Et al., BERT: pre-training of deep bidirectional transformers for lanuage understanding, Proceedings of NAACL-HLT 2019, pp. 4171-4186, (2019)
  • [5] Hu Xu, Bing Liu, Lei Shu, Et al., BERT post-training for review reading comprehension and aspect-based sentiment analysis, Proceedings of NAACL-HLT 2019, pp. 2324-2335, (2019)
  • [6] Yang Liu, Lapata M., Text summarization with pretrained encoders, Proceedings of the 2019 Con-ference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 3730-3740, (2019)
  • [7] Xingxing Zhang, Furu Wei, Ming Zhou, HIBERT: Document level pre-training of hierarchical bidirec-tional transformers for document summarization, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5059-5069, (2019)
  • [8] Nallapati R, Zhou B, Gulcehre C, Et al., Abstractive text summarization using sequence-tosequence RNNs and beyond, Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, pp. 280-290, (2016)
  • [9] Qingyu Zhou, Nan Yang, Furu Wei, Et al., Selective encoding for abstractive sentence summarization, Proceedings of the 55th Annual Meeting of the As-sociation for Computational Linguistics, pp. 1095-1104, (2017)
  • [10] Hsu W T, Lin C K, Lee M Y, Et al., A unified model for extractive and abstractive summarization using inconsistency loss, Proceedings of the 56th Annual Meeting of the Association for Computational Lin-guistics, pp. 132-141, (2018)