Heart failure is a condition where insufficient blood circulation occurs throughout the body, causing the heart to become weak or stiff and not function properly. This can lead to congestive heart failure. The life expectancy of a congestive heart failure patients can vary depending on factors such as disease severity, age, genetics, and other variables. The Centers for Disease Control and Prevention (CDC) has reported that almost half of the patients with congestive heart failure survive for more than five years. Previous studies have attempted to predict the survival rate of heart failure patients, but most of them have been limited by unbalanced data sets and a small number of examples. The main goal of this research is to develop a machine learning model that can accurately predict the survival status of heart failure patients while providing an insight into its decision-making process. Machine learning algorithms are a part of artificial intelligence systems and can be used to predict output values based on input data. However, Many machine learning models lack an explanation of how they arrive at their decisions. Therefore, explainable machine learning algorithms are now being used to understand and explain the decision-making process of machine learning models. In this research, we not only use the latest machine learning algorithms for prediction, but we also incorporate explainability features to understand the results. Hyper-parameter optimization techniques are employed to find the best parameter values for better predictive accuracy. Our data set consists of 1334 heart failure patients' data collected from Felege Hiwot referral hospital and Injibara general hospital. The prediction models used include Decision Tree, Linear Regression, K-Nearest Neighbor, Deep Neural Network, and XGBoost. Prior to conducting the actual experiment, the data set is balanced using the Synthetic Minority Oversampling Technique (SMOTE). Experimental findings indicate that XGBoost performs better than other machine learning techniques, achieving an AUC value of 0.93 using Grid Search HPO after SMOTE.