We address the question of which function spaces are invariant under the action of scalar conservation laws in one and several space dimensions. We establish two types of results. The first result shows that if the initial data is in a rearrangement-invariant function space, then the solution is in the same space for all time. Secondly, we examine which smoothness spaces among the Besov spaces are invariant for conservation laws. Previously, we showed in one dimension that if the initial data has bounded variation and the flux is convex and smooth enough, then the Besov spaces B-q(alpha)(L(q)), alpha > 1, q = 1/(alpha + 1), are invariant smoothness spaces. Now, in one space dimension, we show that no other Besov space with alpha > 1 is invariant. In several space dimensions, we show that no Besov space B-q(alpha)(L(q)) with alpha > 1 is invariant. Combined with previous results, our theorems completely characterize for alpha > 1 which Besov spaces are smoothness spaces for scalar conservation laws.