Deep-Learning-Based Pre-Training and Refined Tuning for Web Summarization Software

被引:0
|
作者
Liu, Mingyue [1 ]
Ma, Zhe [2 ]
Li, Jiale [3 ]
Wu, Ying Cheng [4 ]
Wang, Xukang [5 ]
机构
[1] Cornell Univ, Dept Comp Sci, Ithaca, NY 14850 USA
[2] Univ Southern Calif, Ming Hsieh Dept Elect & Comp Engn, Los Angeles, CA 90007 USA
[3] NYU, Tandon Sch Engn, New York, NY 10012 USA
[4] Univ Washington, Sch Law, Seattle, WA 98195 USA
[5] Sage IT Consulting Grp, Shanghai 200060, Peoples R China
来源
IEEE ACCESS | 2024年 / 12卷
关键词
Pre-training; deep learning; web information extraction;
D O I
10.1109/ACCESS.2024.3423662
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In the digital age, the rapid growth of web information has made it increasingly challenging for individuals and organizations to effectively explore and extract valuable insights from the vast amount of information available. This paper presents a novel approach to automated web text summarization that combines advanced natural language processing techniques with recent breakthroughs in deep learning. we propose a dual-faceted technique that leverages extensive pre-training on a broad dataset outside the domain, followed by a unique refined tuning process. We introduce a carefully curated dataset that captures the heterogeneous nature of web articles and propose an innovative pre-training and tuning approach that establishes a new state-of-the-art in news summarization. Through extensive experiments and rigorous comparisons against existing models, we demonstrate the superiority of our method, particularly highlighting the crucial role of the refined tuning process in achieving these results. Through rigorous experimentation against state-of-the-art models, we demonstrate the superior performance of our approach, highlighting the significance of their refined tuning process in achieving these results.
引用
收藏
页码:92120 / 92129
页数:10
相关论文
共 50 条
  • [1] CBD: A Deep-Learning-Based Scheme for Encrypted Traffic Classification with a General Pre-Training Method
    Hu, Xinyi
    Gu, Chunxiang
    Chen, Yihang
    Wei, Fushan
    [J]. SENSORS, 2021, 21 (24)
  • [2] Statistical-Mechanical Analysis of Pre-training and Fine Tuning in Deep Learning
    Ohzeki, Masayuki
    [J]. JOURNAL OF THE PHYSICAL SOCIETY OF JAPAN, 2015, 84 (03)
  • [3] Fine-tuning and multilingual pre-training for abstractive summarization task for the Arabic language
    Kahla, Mram
    Novak, Attila
    Yang, Zijian Gyozo
    [J]. ANNALES MATHEMATICAE ET INFORMATICAE, 2023, 57 : 24 - 35
  • [4] FACTPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
    Wan, David
    Bansal, Mohit
    [J]. NAACL 2022: THE 2022 CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: HUMAN LANGUAGE TECHNOLOGIES, 2022, : 1010 - 1028
  • [5] Fine-tuning and multilingual pre-training for abstractive summarization task for the Arabic language
    Kahla, Mram
    Novak, Attila
    Yang, Zijian Gyozo
    [J]. ANNALES MATHEMATICAE ET INFORMATICAE, 2023, 57 : 24 - 35
  • [6] Pre-training Fine-tuning data Enhancement method based on active learning
    Cao, Deqi
    Ding, Zhaoyun
    Wang, Fei
    Ma, Haoyang
    [J]. 2022 IEEE INTERNATIONAL CONFERENCE ON TRUST, SECURITY AND PRIVACY IN COMPUTING AND COMMUNICATIONS, TRUSTCOM, 2022, : 1447 - 1454
  • [7] Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers
    Xu, Shusheng
    Zhang, Xingxing
    Wu, Yi
    Wei, Furu
    Zhou, Ming
    [J]. FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, EMNLP 2020, 2020, : 1784 - 1795
  • [8] Supervised pre-training for improved stability in deep reinforcement learning
    Jang, Sooyoung
    Kim, Hyung-Il
    [J]. ICT EXPRESS, 2023, 9 (01): : 51 - 56
  • [9] Why Does Unsupervised Pre-training Help Deep Learning?
    Erhan, Dumitru
    Bengio, Yoshua
    Courville, Aaron
    Manzagol, Pierre-Antoine
    Vincent, Pascal
    Bengio, Samy
    [J]. JOURNAL OF MACHINE LEARNING RESEARCH, 2010, 11 : 625 - 660
  • [10] Pre-training Assessment Through the Web
    Kenneth Wong
    Reggie Kwan
    Jimmy SF Chan
    [J]. 厦门大学学报(自然科学版), 2002, (S1) : 297 - 297