The growth of the gravitational instability associated with a dense layer overlying a lighter layer, Rayleigh-Taylor instability, depends strongly on the constitutive law relating stress and strain rate. We analyse Rayleigh-Taylor instability to understand how the stress dependence of viscosity might affect convective instability associated with the thickening of cold, dense lithosphere beneath mountain belts. The bottom of a layer of dense material, attached to a rigid boundary above it and overlying a fluid with negligible viscosity, is swept into drips or blobs of descending material and thins elsewhere. Numerical calculations for Newtonian fluids corroborate both the exponential growth with time and its dependence on the wavelength of the disturbance predicted by theory for initial growth. Such growth accelerates as the amount of thickening needed to produce the blob reaches 100 per cent of the initial thickness of the layer. With stress-dependent viscosity (strain rate proportional to deviatoric stress raised to a power n), growth is superexponential throughout. The speed, w, at which the bottom of the layer descends, grows with time as: w = [C(n-1)/n)Delta rho g/B h(1-n)(t(b)-t)](n/(1-n)), where C(similar to 1)is an empirical constant that depends on n, wavelength, and the nature of the density distribution (constant in the layer or linearly decreasing with depth), Delta rho is the density difference between the layer and the underlying half-space, B is a measure of resistance to deformation, h is the thickness of the layer, g is gravitational acceleration, t is time, and t(b) is the time at which the blob of the thickened basal part of the layer drops off. Thus, for n>1, w is very small initially and becomes significant as t approaches t,. The time t, can be related to the initial perturbation in thickness of the layer Z(0) by t(b) = (B/g Delta rho h)(n) (N/C)(n) (Z(0)/h((1-n))/(N-1). We exploit these relationships to estimate what fraction of lower lithosphere might be involved in such an instability following its mechanical thickening during mountain building. Following a period in which mountain belts are built by crustal thickening, many such belts collapse by normal faulting and horizontal crustal extension. We use the relationships presented above, assuming that such collapse occurs because removal of thickened mantle lithosphere adds potential energy to the lithosphere of these belts, thereby driving extension by normal faulting. A review of geological studies shows that normal faulting commonly begins where at least a few Myr, and in many cases 30 Myr or more, have elapsed since thickening of the crust ceased. Using Z(0) approximate to h (100 per cent strain) and t(b) = 5, 10 and 20 Myr, we estimated that, for dry olivine, only the bottom 50 to 60 per cent of the lithosphere, that part hotter than 910-950 degrees C, participates in the deblobbing process. For wet olivine, 80 per cent or more of the layer may be removed (T > 750 degrees C). The non-linearity in the constitutive law effects a delay in the convective thinning that is not seen when Newtonian viscosity is assumed.
