We present a combination of a downfolding many-body approach with auxiliary-field quantum Monte Carlo (AFQMC) calculations for extended systems. Many-body calculations operate on a simpler Hamiltonian which retains material-specific properties. The Hamiltonian is systematically improvable and allows one to dial, in principle, between the simplest model and the original Hamiltonian. As a by-product, pseudopotential errors are essentially eliminated using frozen orbitals constructed adaptively from the solid environment. The computational cost of the many-body calculation is dramatically reduced without sacrificing accuracy. Excellent accuracy is achieved for a range of solids, including semiconductors, ionic insulators, and metals. We apply the method to calculate the equation of state of cubic BN under ultrahigh pressure, and determine the spin gap in NiO, a challenging prototypical material with strong electron correlation effects.