The surface step structure of Ga1-xInxAsySb1-y grown by organometallic vapor phase epitaxy on GaSb substrates has been studied by atomic force microscopy. Epilayers were grown at 525 degrees C and 575 degrees C on (001) GaSb substrates misoriented 2 degrees toward (101) or 6 degrees toward (1(1) over bar 1)B. For Ga0.88In0.12As0.1Sb0.9 grown at 575 degrees C, the surface exhibits step-bunching on both types of substrates. When the composition is increased to Ga0.88In0.14As0.12Sb0.88, the periodic step structure breaks down and the surface becomes irregular. The deterioration df the step structure is a consequence of phase separation at the surface of the metastable GaInAsSb epilayer, which leads to the formation of GaAs- and InSb-rich regions. The photoluminescence (PL) of such layers show significant broadening due to carrier recombination in the lower energy gap InSb-rich quaternary regions. On the other hand, the surface of GaInAsSb epilayers grown at a lower temperature of 525 degrees C is vicinal with steps heights of one to two monolayers. The PL FWHM: values are considerably smaller for these layers. This improvement in material quality is related to smaller adatom lifetimes at the lower growth temperature. The importance of surface kinetics as it influences the step structure and thermodynamically driven phase separation is discussed.