Fertilization is an effective approach to improve aggregation of red soil (Ultisol). However, the effect of soil aggregation after long-term organic or mineral fertilization on red soil drought degree during intermittent seasonal drought remains unclear. The objective of this study was to examine the effect of long-term fertilization on soil aggregation (aggregate size distribution and stability) and aggregate-binding agents, and to determine the relationship between aggregation and soil drought degree (D) during wet and dry years. Red soil was continuously applied with manure, NPK + straw, or NPK for 21 years. Surface soil water content (SWC), D, aggregate size distribution, wet aggregate stability, and aggregate-binding agents contents (Fe oxides and soil organic carbon (SOC)) were determined in 2017 (normal wet year), 2019 (dry year), and 2020 (above average wet year). The results showed that manure and NPK + straw treatments resulted in greater SWC variation than the control, especially a sharper SWC decline than control under dry conditions. The results indicated that D values varied with climatic conditions, and that manure and NPK + straw resulted in higher D values than NPK in the normal wet year and dry year, while an opposite trend was observed in the above-average wet year. The D value peaked to 1 in a 70-day dry spell in 2019, while it peaked to 0.7 in < 10-day dry spell in other years. The variation in D can be explained by aggregation due to a significant negative correlation between D and the 2–0.25 mm aggregate size fraction (− 0.58 < r < − 0.64, p < 0.05). In the dry year (2019), the low proportion of 2- to 0.25-mm size aggregates under manure and NPK + straw treatments was significantly correlated with a lower SOC and amorphous Fe oxide content, which might explain their greater D values. Generally, long-term organic fertilization aggravated D during drought periods. The results of this study will contribute to the sustainable management of red soils.