We show that the M-band wavelet transforms of a wide class of covariance matrices consist of subblocks that are essentially banded. Furthermore, we prove that the Cholesky factors of the transformed covariance matrices also consist of subblocks that are essentially banded. We combine these two observations to construct a fast O(N2) algorithm for solving the N x N linear positive definite systems of equations that arise in statistical signal processing. Finally, we provide an error analysis of the proposed linear positive definite system solver.