BEHM-GAN: Bandwidth Extension of Historical Music Using Generative Adversarial Networks

被引:4
|
作者
Moliner, Eloi [1 ]
Valimaki, Vesa [1 ]
机构
[1] Aalto Univ, Dept Informat & Commun Engn, Acoust Lab, Espoo 02150, Finland
关键词
Recording; Bandwidth; Training; Task analysis; Speech processing; Hidden Markov models; Cutoff frequency; Audio recording; convolutional neural networks; machine learning; music; signal restoration; NEURAL-NETWORK;
D O I
10.1109/TASLP.2022.3190726
中图分类号
O42 [声学];
学科分类号
070206 ; 082403 ;
摘要
Audio bandwidth extension aims to expand the spectrum of bandlimited audio signals. Although this topic has been broadly studied during recent years, the particular problem of extending the bandwidth of historical music recordings remains an open challenge. This paper proposes a method for the bandwidth extension of historical music using generative adversarial networks (BEHM-GAN) as a practical solution to this problem. The proposed method works with the complex spectrogram representation of audio and, thanks to a dedicated regularization strategy, can effectively extend the bandwidth of out-of-distribution real historical recordings. The BEHM-GAN is designed to be applied as a second step after denoising the recording to suppress any additive disturbances, such as clicks and background noise. We train and evaluate the method using solo piano classical music. The proposed method outperforms the compared baselines in both objective and subjective experiments. The results of a formal blind listening test show that BEHM-GAN significantly increases the perceptual sound quality in early-20th-century gramophone recordings. For several items, there is a substantial improvement in the mean opinion score after enhancing historical recordings with the proposed bandwidth-extension algorithm. This study represents a relevant step toward data-driven music restoration in real-world scenarios.
引用
收藏
页码:943 / 956
页数:14
相关论文
共 50 条
  • [1] SPEECH BANDWIDTH EXTENSION USING GENERATIVE ADVERSARIAL NETWORKS
    Li, Sen
    Villette, Stephane
    Ramadas, Pravin
    Sinder, Daniel J.
    [J]. 2018 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2018, : 5029 - 5033
  • [2] αβ-GAN: Robust generative adversarial networks
    Aurele Tohokantche, Aurele Tohokantche
    Cao, Wenming
    Mao, Xudong
    Wu, Si
    Wong, Hau-San
    Li, Qing
    [J]. INFORMATION SCIENCES, 2022, 593 : 177 - 200
  • [3] ARTIFICIAL BANDWIDTH EXTENSION USING A CONDITIONAL GENERATIVE ADVERSARIAL NETWORK WITH DISCRIMINATIVE TRAINING
    Sautter, Jonas
    Faubel, Friedrich
    Buck, Markus
    Schmidt, Gerhard
    [J]. 2019 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2019, : 7005 - 7009
  • [4] Augmenting Historical Alphabet Datasets Using Generative Adversarial Networks
    Franc, David
    Hamplova, Adela
    Svojse, Ondrej
    [J]. DATA SCIENCE AND ALGORITHMS IN SYSTEMS, 2022, VOL 2, 2023, 597 : 132 - 141
  • [5] Boundless: Generative Adversarial Networks for Image Extension
    Teterwak, Piotr
    Sarna, Aaron
    Krishnan, Dilip
    Maschinot, Aaron
    Belanger, David
    Liu, Ce
    Freeman, William T.
    [J]. 2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, : 10520 - 10529
  • [6] Historical Document Synthesis With Generative Adversarial Networks
    Pondenkandath, Vinaychandran
    Alberti, Michele
    Diatta, Michael
    Ingold, Rolf
    Liwicki, Marcus
    [J]. 2019 INTERNATIONAL CONFERENCE ON DOCUMENT ANALYSIS AND RECOGNITION WORKSHOPS (ICDARW), VOL 5, 2019, : 146 - 151
  • [7] WSD-GAN: Word Sense Disambiguation Using Generative Adversarial Networks
    Hu, Zijian
    Luo, Fuli
    Tan, Yutong
    Zeng, Wenxin
    Sui, Zhifang
    [J]. THIRTY-THIRD AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FIRST INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE / NINTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2019, : 9943 - 9944
  • [8] PL-GAN: Path Loss Prediction Using Generative Adversarial Networks
    Marey, Ahmed
    Bal, Mustafa
    Ates, Hasan F.
    Gunturk, Bahadir K.
    [J]. IEEE ACCESS, 2022, 10 : 90474 - 90480
  • [9] Ret-GAN: Retinal Image Enhancement using Generative Adversarial Networks
    Santosh, K. C.
    Ghosh, Sourodip
    Bose, Moinak
    [J]. 2021 IEEE 34TH INTERNATIONAL SYMPOSIUM ON COMPUTER-BASED MEDICAL SYSTEMS (CBMS), 2021, : 79 - 84
  • [10] FV-GAN: Finger Vein Representation Using Generative Adversarial Networks
    Yang, Wenming
    Hui, Changqing
    Chen, Zhiquan
    Xue, Jing-Hao
    Liao, Qingmin
    [J]. IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, 2019, 14 (09) : 2512 - 2524