We analyze the effect of solar features on the variability of the solar irradiance in three different spectral ranges. Our study is based on two solar-cycles' worth of full-disk photometric images from the San Fernando Observatory, obtained with red, blue, and Ca II K-line filters. For each image we measure the photometric sum, Sigma, which is the relative contribution of solar features to the disk-integrated intensity of the image. The photometric sums in the red and blue continuum, Sigma(r) and Sigma(b), exhibit similar temporal patterns: they are negatively correlated with solar activity, with strong short-term variability, and weak solar-cycle variability. However, the Ca II K-line photometric sum, Sigma(K), is positively correlated with solar activity and has strong variations on solar-cycle timescales. We show that we can model the variability of the Sun's bolometric flux as a linear combination of Sigma(r) and Sigma(K). We infer that, over solar-cycle timescales, the variability of the Sun's bolometric irradiance is directly correlated with spectral line variability, but inversely correlated with continuum variability. Our blue and red continuum filters are quite similar to the Stromgren b and y filters used to measure stellar photometric variability. We conclude that active stars whose visible continuum brightness varies inversely with activity, as measured by the Ca HK index, are displaying a pattern that is similar to that of the Sun, i.e., radiative variability in the visible continuum that is spot-dominated.