A Hybrid Regression Model for Improving Prediction Accuracy

Regression Tree (RT) and K-Nearest Neighbor (KNN) models play significant roles in machine learning. RT facilitates interpretable decision-making, aiding in the comprehension of complex data relationships, while KNN is valued for its simplicity, adaptability to non-linear data, and robustness to noise, making it a versatile tool across various applications. The primary drawback of Regression Tree is its tendency to assign the same predicted value (average value) to all tuples satisfying the same corresponding splitting criterion. K-Nearest Neighbors (KNN) is sensitive to irrelevant or redundant features since all features contribute to similarity. This paper proposes a hybrid regression model based on Regression Tree (RT) and KNN, addressing the aforementioned issues. The model's performance is compared with KNN using 10 types of distance measures and further assessed against RT, K Nearest Neighbor regression (KNN), and Support Vector Regression (SVR) through a Monte Carlo simulation study. Simulation results indicate that the hybrid model outperforms all other regression models, regardless of sample size, when observations follow normal distributions or t-distributions.The proposed model's effectiveness is demonstrated through a real-life application using data on global warming in Delhi.