We study the statistics of the condition number kappa = lambda(max)/lambda(min) (the ratio between largest and smallest squared singular values) of N x M Gaussian random matrices. Using a Coulomb fluid technique, we derive analytically and for large N the cumulative P(kappa < x) and tail-cumulative P(kappa > x) distributions of kappa. We find that these distributions decay as P(kappa < x) approximate to exp[-beta N-2 Phi(-)(x)] and P(kappa > x) approximate to exp[-beta N Phi(+)(x)], where beta is the Dyson index of the ensemble. The left and right rate functions Phi(+/-) (x) are independent of beta and calculated exactly for any choice of the rectangularity parameter alpha = M/N - 1 > 0. Interestingly, they show a weak nonanalytic behavior at their minimum <kappa > (corresponding to the average condition number), a direct consequence of a phase transition in the associated Coulomb fluid problem. Matching the behavior of the rate functions around <kappa >, we determine exactly the scale of typical fluctuations similar to O(N-2/3) and the tails of the limiting distribution of kappa. The analytical results are in excellent agreement with numerical simulations.