Renormalization group and spontaneous compactification of a higher-dimensional scalar field theory in curved spacetime

The renormalization group (RG) is used to study the asymptotically free phi(6)(3) theory in curved spacetime. Several forms of the RG equations for the effective potential are formulated. By solving these equations we obtain the one-loop effective potential as well as its explicit forms in the case of strong gravitational fields and strong scalar fields. Using zeta-function techniques, the one-loop vacuum energy and corresponding RG-improved vacuum energy are found for the Kaluza-Klein backgrounds R(4)XS(1)XS(1) and R(4)XS(2). They are given in terms of exponentially convergent series, appropriate for numerical calculations. A study of these vacuum energies as a function of compactification lengths and other couplings shows that spontaneous compactification can be qualitatively different when the RG-improved energy is used.