Rapid urbanization and population growth are the main problems faced by developing countries that lead to natural resource depletion in the periphery of the city. This research attempts to analyze the impacts of urban land use land cover (LULC) change on land surface temperature (LST) from 1991 to 2021 in Jimma city, southwestern Ethiopia. Landsat Thematic Mapper (TM) 1991, Landsat Enhanced Thematic Mapper Plus (ETM +) 2005, and Landsat-8 Operational land imagery (OLI)/Thermal Infrared Sensor (TIRS) 2021 were used in this study. Multispectral bands and thermal infrared bands of Landsat images were used to calculate LULC change, normalized difference vegetation index (NDVI), normalized difference built-up index (NDBI), and LST. The LULC of the study area was classified using a supervised classification method with the maximum likelihood algorithm. The results of this study clearly showed that there is a negative correlation between vegetation cover and LST. The decrease in vegetation coverage and expansion of impervious surfaces lead to elevated LST in urban areas. The loss of vegetation cover contributed to the increasing trend of LST. Moreover, the conversion of vegetation cover to impervious surfaces aggravates the problem of LST. The results revealed that the built-up area was increased at a rate of 0.4 km2/year from 1991 to 2021. The vegetation cover in the city declined due to urban expansion to the periphery of the city. Consequently, the dense vegetation and sparse vegetation were converted into built-up areas by approximately 5.2 km2 during the study period. The mean LST of the study area increased by 10.3 °C from 1991 to 2021 during the winter season in daytime. To improve the problems of climate change around urban areas, all stakeholders should work together to increase the urban green space coverage, which will contribute a significant role in mitigating LST and the urban heat island effect. More specifically, all residents could be accessible to public green spaces around big cities.