Evaluating content selection in summarization: The pyramid method

被引：0
|
作者：
Nenkova, A [1 ]
Passonneau, R [1 ]
机构：
[1] Columbia Univ, Dept Comp Sci, New York, NY 10027 USA
来源：
HLT-NAACL 2004: HUMAN LANGUAGE TECHNOLOGY CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, PROCEEDINGS OF THE MAIN CONFERENCE | 2004年
关键词：
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
We present an empirically grounded method for evaluating content selection in summarization. It incorporates the idea that no single best model summary for a collection of documents exists. Our method quantifies the relative importance of facts to be conveyed. We argue that it is reliable, predictive and diagnostic, thus improves considerably over the shortcomings of the human evaluation method currently used in the Document Understanding Conference.
引用
收藏
页码：145 / 152
页数：8
相关论文
共 50 条
  • [1] The Pyramid Method: Incorporating human content selection variation in summarization evaluation
    Nenkova, Ani
    Passonneau, Rebecca
    Mckeown, Kathleen
    ACM Transactions on Speech and Language Processing, 2007, 4 (02):
  • [2] An Entailment-based Scoring Method for Content Selection in Document Summarization
    Dang Hoang Long
    Minh-Tien Nguyen
    Ngo Xuan Bach
    Le-Minh Nguyen
    Tu Minh Phuong
    PROCEEDINGS OF THE NINTH INTERNATIONAL SYMPOSIUM ON INFORMATION AND COMMUNICATION TECHNOLOGY (SOICT 2018), 2018, : 122 - 129
  • [3] A behavior mode for content unit selection in summarization
    Teng, Chong
    He, Yanxiang
    Liu, Dexi
    Ji, Donghong
    Yang, Hua
    Xiong, Naixue
    RECENT ADVANCE OF CHINESE COMPUTING TECHNOLOGIES, 2007, : 250 - 254
  • [4] Content Selection in Deep Learning Models of Summarization
    Kedzie, Chris
    McKeown, Kathleen
    Daume, Hal, III
    2018 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2018), 2018, : 1818 - +
  • [5] Content-selection based video summarization
    Chang, I. -Cheng
    Chen, Kun-You
    ICCE: 2007 DIGEST OF TECHNICAL PAPERS INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS, 2007, : 399 - +
  • [6] Pyramid Attention For Source Code Summarization
    Chai, Lei
    Li, Ming
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35 (NEURIPS 2022), 2022,
  • [7] Attend to Medical Ontologies: Content Selection for Clinical Abstractive Summarization
    Sotudeh, Sajad
    Goharian, Nazli
    Filice, Ross W.
    58TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2020), 2020, : 1899 - 1905
  • [8] A Few Good Sentences: Content Selection for Abstractive Text Summarization
    Srivastava, Vivek
    Bhat, Savita
    Pedanekar, Niranjan
    MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES: RESEARCH TRACK, ECML PKDD 2023, PT IV, 2023, 14172 : 124 - 141
  • [9] A Cascade Approach to Neural Abstractive Summarization with Content Selection and Fusion
    Lebanoff, Logan
    Dernoncourt, Franck
    Kim, Doo Soon
    Chang, Walter
    Liu, Fei
    1ST CONFERENCE OF THE ASIA-PACIFIC CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS AND THE 10TH INTERNATIONAL JOINT CONFERENCE ON NATURAL LANGUAGE PROCESSING (AACL-IJCNLP 2020), 2020, : 529 - 535
  • [10] A Method for Evaluating Modern Systems of Automatic Text Summarization
    Yatsko, V. A.
    Vishnyakov, T. N.
    AUTOMATIC DOCUMENTATION AND MATHEMATICAL LINGUISTICS, 2007, 41 (03) : 93 - 103
12345