Relative perturbation theory: I. Eigenvalue and singular value variations

The classical perturbation theory for Hermitian matrix eigenvalue and singular value problems provides bounds on the absolute differences between approximate eigenvalues (singular values) and the true eigenvalues (singular values) of a matrix. These bounds may be bad news for small eigenvalues (singular values), which thereby suffer worse relative uncertainty than large ones. However, there are situations where even small eigenvalues are determined to high relative accuracy by the data much more accurately than the classical perturbation theory would indicate. In this paper, we study how eigenvalues of a Hermitian matrix A change when it is perturbed to A = D* AD, where D is close to a unitary matrix, and how singular values of a (nonsquare) matrix B change when it is perturbed to (B) over tilde = D-1* BD2, where D-1 and D-2 are nearly unitary. It is proved that B under these kinds of perturbations small eigenvalues (singular values) suffer relative changes no worse than large eigenvalues (singular values). Many well-known perturbation theorems, including the Hoffman-Wielandt and Weyl-Lidskii theorems, are extended.