Climate change and its impacts continue and become a major challenge if proper mitigation measures are not taken. This study aimed to analyze and predict the impact of climate change on crop yield under two Representative Concentration Pathways (RCPs), high (RCP8.5) and medium (RCP4.5) emission global warming scenarios of rainfall and temperature data derived from the Coordinated Regional Climate Downscaling Experiment (CORDEX) Africa, Regional Climate Models (RCMs). The performance of REgional MOdel (REMO2009), High-Resolution Hamburg Climate Model 5 (HIRAM5), Climate Limited-Area Modeling Community (CCLM4-8), and Rossby Center Regional Atmospheric Model (RCA4) RCMs were evaluated using observed rainfall and temperature data of (1985–2005), and the results proved that the ensemble of these models performed well as compared to the performance of individual models. Bias-corrected RCMs ensemble rainfall and temperature projected data of RCP8.5 and RCP4.5 for the future period of (2030–2050) were used as an input in artificial neural network (ANN) to predict sorghum, maize, teff, and wheat yield. Analysis of the projected data shows that under RCP8.5 scenarios, the maximum temperature will increase up to 2.84 °C while the minimum temperature is expected to increase in the range of 1.36–2.2 °C. Under RCP4.5, the minimum temperature increases in the range of 0.38–1.83 °C. The projected rainfall shows that there would be a shift in maximum rainfall from July to August. Under RCP8.5 temperature scenarios, the sorghum, wheat, and teff yield would decrease within the range of 2.64–8.42%, 4.47–8.35%, and 1.77–9.77% respectively. In the RCP8.5 rainfall scenario, maize and wheat yield are expected to increase, but teff and sorghum to decrease. In the future period of 2030–2050 years, the change in temperature will have a greater impact on crop yield than the change in rainfall.