Uniform material removal affects the profile accuracy and surface quality of the optical surface in polishing. In general, polishing path planning and material removal optimization are studied separately. In this paper, the research combining these two aspects is employed to better ensure material removal uniformity and polishing efficiency. Two common polishing paths are covered: the scanning path and the concentric circle path. The scanning path is divided into the inner area and edge area, and the concentric circle path is divided into the peripheral area and central area, respectively. Material removal caused by the polishing path is analyzed by the simulation of the material removal map. For the scanning path, the spacing range is defined according to the number of superimposed removal profiles on the cross-section of the path, and the optimal spacing value in each spacing range is determined by using the numerical method. Then, the feed velocity is optimized to control the material removal thickness in the inner area, eliminate the over-polishing in the edge area, and ensure the polishing efficiency. For the concentric circle path, in the peripheral area, the influence of the path curvature on the removal profile is eliminated by optimizing the inclination angle and feed velocity, so that the spacing law of the scanning path can be applied to the concentric circle path. Furthermore, the over-polishing in the central area is eliminated by optimizing the radius of the central path, inclination angle, and feed velocity. Simulations and experiments are conducted to verify the feasibility of the proposed method. The results show that the proposed method can effectively improve the material removal uniformity, control the material removal thickness, and ensure the polishing efficiency.