The problem of continuous quantum phase transitions in metals involves critical bosons coupled to a Fermi surface. We solve the theory in the limit of a large number, N-B, of bosonic flavors, where the bosons transform in the adjoint representation (a matrix representation), while the fermions are in the fundamental representation (a vector representation) of a global SU(N-B) flavor symmetry group. The leading large N-B solution corresponds to a non-Fermi liquid coupled to Wilson-Fisher bosons. In a certain energy range, the fermion velocity vanishes-resulting in the destruction of the Fermi surface. Subleading 1/N-B corrections correspond to a qualitatively different form of Landau damping of the bosonic critical fluctuations. We discuss the model in d = 3 - epsilon but because of the additional control afforded by large N-B, our results are valid down to d = 2. In the limit epsilon << 1, the large N-B solution is consistent with the renormalization group analysis of Fitzpatrick et al. [Phys. Rev. B 88, 125116 (2013)].