We present Hubble Space Telescope (HST) GHRS spectra of Mg II absorption from our own Galaxy toward seven extragalactic background sources. Along four sight lines, the bulk of the absorption occurs at velocities consistent with those expected for halo gas corotating with the disk of the Galaxy. In the other three directions the line profiles extend to peculiar velocities, differing from 0 to 50 km s(-1) from corotating gas. Only in one direction-toward Q1219+755, originally discussed by Bowen and Blades (1993)-do we find Mg II absorption at velocities greater than 100 km s(-1), characteristic of high-velocity clouds (HVCs). The remaining six lines of sight show no absorption to 2 sigma equivalent width limits of 51-73 m Angstrom, corresponding to column densities N(Mg II) similar or equal to 2 x 10(12) cm(-2). For plausible ionization conditions these limits correspond to H I column densities N(H I) similar to 10(16-17) cm(-2), approximate to 1-2 dex less than those detectable from 21 cm emission. We conclude that the covering factor of HVCs at these low levels is similar to 14% +/- 14%, not as high as previously speculated. We find no HVCs along any sight line in our sample with a column density high enough to produce Mg II lines detectable with both GHRS and FOS. This is in marked contrast with the results of Savage et al. (1993) who found absorbing HVCs toward seven of the 15 QSO sight lines they observed with the FOS. However, we show that by adopting similar selection procedures, the two data sets yield covering factors which differ at only the similar to 2 sigma level, a result which is not unexpected given the known clumping of HVCs. By combining the two sets of data we deduce that HVCs capable of producing Mg II absorption with W(lambda 2796) greater than or similar to 0.4 Angstrom cover similar to 14% +/- 8% of the sky. A literature search for 21 cm emission in directions close to our sight lines show that there are HVCs within approximate to 1 degrees-4 degrees of five of the background probes. We interpret the fact that these HVCs do not produce Mg II absorption in our spectra as evidence that HVCs are sharply bounded, probably by ionization; this would lead to a natural discontinuity in the covering factor of HVCs at N(H I) similar to 10(17) cm(-2).