We discuss the low-temperature specific heat of the integrable SU(N)-invariant Heisenberg model in one dimension with degrees of freedom in the symmetric rank-m tensor representation, especially for the antiferromagnetic coupling. It is known that the linear specific heat coefficient gamma[N,m] is a function of a field which breaks the SU(N) invariance of the internal degrees of freedom. We calculate the gamma[N,m] in zero field and in a small field. The in-field gamma[N,m] is less than the zero-field gamma[N,m] as expected since the entropy is reduced in the ordered system. The zero-field gamma[N,m] is the same as the one obtained by the prediction of the critical behavior of ID quantum spin systems via conformal field theory. This extends the previous results for N = 2 to arbitrary N.