A Semi-Randomized and Augmented Kaczmarz Method With Simple Random Sampling for Large-Scale Inconsistent Linear Systems

A greedy randomized augmented Kaczmarz (GRAK) method was proposed in Bai and Wu for large and sparse inconsistent linear systems. However, in each iteration of this method, one has to construct two new index sets via computing residual vector with respect to the augmented linear system. Thus, the computational overhead of this method may be large for extremely large-scale problems. Moreover, there is no reliable stopping criterion for this method. In this work, we are interested in solving large-scale sparse or dense inconsistent linear systems, and try to enhance the numerical performance of this method. The contributions of this work are as follows. First, we propose an accelerated GRAK method. Theoretical analysis indicates that it converges faster than the GRAK method under a very weak assumption. Second, we apply the semi-randomized Kaczmarz method to the augmented linear system, and propose a semi-randomized augmented Kaczmarz method with simple random sampling. In this method, there is no need to access all the information of the data matrix. The convergence of the proposed methods are established. To the best of our knowledge, there are no practical stopping criteria in all randomized Kaczmarz-type methods till now. To fill-in this gap, the third contribution of this work is to introduce a practical stopping criterion for Kaczmarz-type methods, and show its rationality from a theoretical point of view. Numerical results are performed on both real-world and synthetic data sets, which demonstrate the effectiveness of the proposed stopping criterion, and illustrate that the new methods are often superior to many state-of-the-art Kaczmarz methods for large-scale inconsistent linear systems.