Many studies on global climate have forecast major changes in the amounts and spatial patterns of precipitation that may significantly affect temperate grasslands in arid and semi-arid regions. As a part of ChinaFLUX, eddy covariance flux measurements were made at a semi-arid Leymus chinensis steppe in Inner Mongolia, China during 2003–2004 to quantify the response of carbon exchange to environmental changes. Results showed that gross ecosystem production (FGEP) and ecosystem respiration (Reco) of the steppe were significantly depressed by water stress due to lack of precipitation during the growing season. Temperature was the dominant factor affecting FGEP and Reco in 2003, whereas soil moisture imposed a significant influence on both Reco and FGEP in 2004. Under wet conditions, Reco showed an exponentially increasing trend with temperature (Q10 = 2.0), but an apparent reduction in the value of Reco and its temperature sensitivity were observed during the periods of water stress (Q10=1.6). Both heat and water stress can cause decrease in FGEP. The seasonality of ecosystem carbon exchange was strongly correlated with the variation of precipitation. With less precipitation in 2003, the steppe sequestrated carbon in June and July, and went into a senescence in early August due to water stress. As compared to 2003, the severe drought during the spring of 2004 delayed the growth of the steppe until late June, and the steppe became a CO2 sink from early July until mid-September, with ample precipitation in August. The semi-arid steppe released a total of 9.7 g C·m−2 from May 16 to the end of September 2003, whereas the net carbon budget during the same period in 2004 was close to zero. Long-term measurements over various grasslands are needed to quantify carbon balance in temperate grasslands.