On Monte Carlo Methods in Distributed Memory Systems

The way for solving a system of linear algebraic equations (SLAEs) with computers with distributed memory is presented. It is assumed that there are M computing nodes, each of which has a limited fast memory, and communication between nodes takes considerable time. If the matrix elements and the right side vectors cannot be placed in their entirety in the one node memory, the problem of using equipment efficiently between the exchange, i.e., whether each node is able to use the available information to reduce the total residual, appears. The answer to this question is negative under general assumptions on the system's matrix and the example presented in the Appendix verifies this fact. We examine the case when the system is of sufficiently high order and it is reasonable to use the Monte Carlo method. In this case the matrix is divided between computing nodes on blocks of rows that do not overlap with the same partition into blocks of indices of rows and columns. We also consider a modification of the method of simple iteration based on this partition consisting of two nested iterative processes so that messaging between nodes takes place only in the outer iterations. This iterative process naturally results in a similar process, where the Monte Carlo method is used, and where it is not necessary to save a matrix's full copy at each computing node. The unbiased estimations of linear algebraic equations' solutions for the examined case are studied in the present paper. Under certain additional conditions imposed on the matrix, we prove the sufficient convergence conditions.