This study explores the role of residual halide ions (Cl, Br, and I) in modulating the optical and electronic properties of heterostructured InP quantum dots (QDs) with ZnSe/ZnS double shells. By synthesizing halide-containing InP cores using aminophosphine chemistry, we investigate the impact of surface halides on the energy levels of red-, amber-, and green-emissive InP QDs. X-ray photoelectron and ultraviolet photoelectron spectroscopic analyses confirm the presence of halides on the InP core surface, which induces surface dipole and shift energy levels. Our findings reveal that the interfacial Cl and Br ions cause significant alterations in the conduction and valence band energy levels, resulting in band gap reduction and a photoluminescence (PL) red shift in the heterostructured QDs. These effects are the most pronounced for Cl-containing red-emissive QDs, while green-emissive QDs with I ions exhibit negligible changes. These results provide new insights into how surface halide ligands at the core-shell interface influence the optical performance of InP-based heterostructures, offering potential pathways for tuning the properties of QDs for advanced optoelectronic applications.