Using high-precision numerical analysis, we show that 3 + 1 dimensional gauge theories holographically dual to 4 + 1 dimensional Einstein-Maxwell-Chern-Simons theory undergo a quantum phase transition in the presence of a finite charge density and magnetic field. The quantum critical theory has dynamical scaling exponent z = 3, and is reached by tuning a relevant operator of scaling dimension 2. For magnetic field B above the critical value B-c, the system behaves as a Fermi liquid. As the magnetic field approaches B c from the high field side, the specific heat coefficient diverges as 1/(B - B-c), and non-Fermi liquid behavior sets in. For B < B-c the entropy density s becomes non-vanishing at zero temperature, and scales according to s similar to root B-c - B. At B = B-c, and for small non-zero temperature T, a new scaling law sets in for which s similar to T-1/3. Throughout a small region surrounding the quantum critical point, the ratio s/T-1/3 is given by a universal scaling function which depends only on the ratio (B - B-c)/T-2/3. The quantum phase transition involves non-analytic behavior of the specific heat and magnetization but no change of symmetry. Above the critical field, our numerical results are consistent with those predicted by the Hertz/Millis theory applied to metamagnetic quantum phase transitions, which also describe non-analytic changes in magnetization without change of symmetry. Such transitions have been the subject of much experimental investigation recently, especially in the compound Sr3Ru2O7, and we comment on the connections.