We consider surfaces with constant mean curvature in certain warped product manifolds. We show that any such surface is umbilic, provided that the warping factor satisfies certain structure conditions. This theorem can be viewed as a generalization of the classical Alexandrov theorem in Euclidean space. In particular, our results apply to the deSitter-Schwarzschild and Reissner-Nordstrom manifolds.