The rapid outbreak of COVID-19 has proven to be a dangerous virus with catastrophic effects on large populations and health systems worldwide. Therefore, in order to limit the rapid spread of this virus, artificial intelligence (AI) combined with radiological images such as chest X-rays (CXRs) has recently become a worthwhile option for screening COVID-19 patients, especially in the early stages. We suggest a solution to address the given problem by using a stacked ensemble model. This model combines the predictions of multiple individual models, resulting in improved accuracy compared to using each model separately. Fourteen well-known network architectures (VGG, DenseNet, InceptionResNetV2, ResNetV2 (50, 101, 152), InceptionV3, NasNetMobile, Xception and MobileNet) were trained and evaluated using two forms of transfer learning (TL) strategies, namely feature extraction and fine-tuning. We build network architectures by replacing the original ImageNet classifier with our classifier head, consisting of dense, batch normalization, dropout, and a softmax layer. The experiments conducted indicate that fine-tuning the higher layers of pre-trained architectures can provide more detailed and informative features compared to using "off-the-shelf" features, ultimately resulting in improved classification performance. To boost the classification performance, we utilized a stack ensemble technique that involved combining the prediction scores of the four top performing fine-tuned models: VGG19, DenseNet169, MobileNet, and DenseNet201. By employing this technique, we were able to obtain a robust ensemble model that significantly improved the performance. For model interpretability, feature maps and Grad-CAM analysis are performed to visualize the feature learning procedures that are significant for prediction. For experiments, the research work analyzed two CXR datasets that are very common for detecting COVID-19. The ensemble architecture yielded the highest classification accuracy of 99.03% for the 3-class classification and 99.02% for the 4-class classification. The experimental analysis revealed that the proposed ensemble architecture outperforms existing methods in classifying COVID-19 patients, offering greater accuracy and potential for assisting radiologists with improved screening efficiency.