We extend the existing quasiclassical theory for the superconducting proximity effect in a chaotic quantum dot, to include a time-reversal-symmetry breaking magnetic field. Random-matrix theory (RMT) breaks down once the Ehrenfest time tau(E) becomes longer than the mean time tau(D) between Andreev reflections. As a consequence, the critical field at which the excitation gap closes drops below the RMT prediction as tau(E)/tau(D) is increased. Our quasiclassical results are supported by comparison with a fully quantum mechanical simulation of a stroboscopic model (the Andreev kicked rotator).