Electrocardiogram (ECG) is susceptible to different kinds of noises whose removal is necessary for accurate clinical diagnosis. This paper proposes a hybrid technique that integrates the concepts of sparsity, wavelet transform, and extreme learning machine into a single framework. Initially, the loss function of the sparsity-based method is designed with linear time-variant filtering parameters, and a compound penalty-based Huber function is used for the removal of low-frequency baseline wander. Sparse optimization is carried out by the majorization-minimization (MM) technique ensuring fast and guaranteed convergence irrespective of initialization. The next step involves wavelet-based de-noising with novel thresholding followed by extreme machine learning for remnant noise removal. The comparative analysis of the proposed method is done on the MIT-BIH Arrhythmia database for baseline wander (BW), additive white Gaussian noise (AWGN), muscle artifacts (MA), power-line interference (PLI), and composite noise (CN) both qualitatively and quantitatively. Qualitative analysis is also performed on MIT-BIH NSR and MIT-BIH NST. For AWGN, BW, MA, PLI, and CN, SNRimp is maximum at 27.8670 dB (record 119), 32.5962 dB (record 215), 25.7825 dB (record 119), 31.9277 dB (record 215), 25.5463 dB (record 105) at an SNRin of 10 dB respectively. Significant improvement in terms of SNRimp, RMSE, and PRD is obtained over the state-of-the-art ECG de-noising methods. Feature preservation in the de-noised ECG signal is also investigated with the help of fiducial morphological features. (c) 2022 Elsevier B.V. All rights reserved.