We study the critical window of the symmetric binary perceptron, or equivalently, random combinatorial discrepancy. Consider the problem of finding a +/- 1-valued vector sigma satisfying ||A sigma||(infinity) <= K, where A is an alpha n x n matrix with iid Gaussian entries. For fixed K, at which constraint densities ff is this constraint satisfaction problem (CSP) satisfiable? A sharp threshold was recently established by Perkins and Xu [29], and Abbe, Li, and Sly [2], answering this to first order. Namely, for each K there exists an explicit critical density alpha(c) so that for any fixed epsilon > 0, with high probability the CSP is satisfiable for alpha n < (alpha(c) - epsilon)n and unsatisfiable for alpha n > (alpha(c) + epsilon)n. This corresponds to a bound of o(n) on the size of the critical window. We sharpen these results significantly, as well as provide exponential tail bounds. Our main result is that, perhaps surprisingly, the critical window is actually at most of order log(n). More precisely, for a large constant C, with high probability the CSP is satisfiable for alpha n < alpha(c)n - C log(n) and unsatisfiable for alpha n > alpha(c)n + C. These results add the the symmetric perceptron to the short list of CSP models for which a critical window is rigorously known, and to the even shorter list for which this window is known to have nearly constant width.