We provide evidence on the impacts of climate change on African agriculture using recent multi-country farmer microdata and representative concentration pathways (RCP) adopted by the IPCC. Climate change has direct implications on agriculture globally but especially in sub-Saharan Africa where many small-scale farmers are dependent on it for their livelihoods. We quantify the impact of climate change on farmers by leveraging data from a coordinated survey across six countries: Burkina Faso, São Tomé and Príncipe, Sierra Leone, Uganda, Tanzania and Zambia. We group our sample into an eastern African subsample i.e., Tanzania, Uganda and Zambia and a western African subsample i.e., Burkina Faso, São Tomé and Príncipe and Sierra Leone. Our analysis suggests that the western African group of countries is more sensitive to changes in temperature and precipitation, with summer temperature increases being particularly damaging. We extend our analysis to evaluate the effects of future climate scenarios (RCPs 2.6, 4.5, 8.5) over the short-, medium-, and long-term (2021–2040, 2041–2060, 2081–2100). We find that a changing climate damages farms in both the eastern and western African subsamples, increasing with the severity of the climate change scenario and over time, especially the latter part of the century. We also find significant heterogeneity in impacts with farms in the western group likely to suffer considerably more than those in the eastern group. We conclude by recommending that agricultural policy in Africa focus on assisting farmers in adapting to climate change given its expected near-term impacts and that the heterogeneity in impacts we observe be leveraged to help mitigate damages.