Rapid urbanization has increased the demand for energy in various sectors. Previous studies have focused more on individual sectors when investigating the spatial impact of urbanization on CO2, but few have compared the differences among sectors. This study uses geographically weighted regression model to investigate the effect of urbanization on CO2 emissions in six sectors in 2000, 2005, 2010, and 2015. Industrial structure and energy intensity were selected as control variables in this study. The results indicate that the impact of urbanization on CO2 emissions in agriculture, forestry, animal husbandry, fishery and water conservancy sector declines from the southeastern coastal area to the northwest. The influence of urbanization on CO2 emissions presents an increasing trend from the northeast region to the southwest region in the construction industry sector. In industry and residential consumption sectors, urbanization had a greater impact on CO2 emissions in the eastern region than in the western region. However, urbanization had a stronger impact on CO2 emissions in the western region than in the eastern region in the transportation, storage, and postal industry sector. As for wholesale and retail trade and catering and accommodation industry sector, the effect of urbanization on CO2 emissions in the southern region was larger than in the northern region. To achieve the goal of ultimately reducing carbon emissions, policymakers should formulate differential policy for different sectors in different regions, optimize the industrial structure, and develop high technology.