Distributed-Shared Memory Computed Tomography

Large-scale statistical reconstruction algorithms are known to be memory and processor intensive applications. For instance, the system matrix for a small animal scanner requires several gigabytes of memory storage and the algorithm usually needs many iterations to produce acceptable images. In this abstract we design distributed-shared memory (DSM) statistical reconstruction algorithms to exploit all available computational resources as a unified infrastructure and thereby improving the cost-efficiency of the investment and scalability of the system. We use and compare two distinct approaches. The first one uses the Unified Parallel C (UPC) compiler which transparently provides a global shared virtual address space across all computers. Data is physically stored in different computers, but threads can access any shared item as it if were in its local memory. The second approach combines OpenMP and Pthreads shared-memory libraries with the message-passing library MPI. In this case threads only have access to the node's local memory and access to remote data is carried out explicitly through message-passing. Early UPC experiments showed that keeping all data shared heavily affects reconstruction performance. Therefore, we devised a distribution method where some data is kept shared and other is kept private, mimicking somehow the library-based approach. However, even with data privatization, the compiler solution cannot compete with the library solutions. We explore three workload distribution strategies: LOR-based, Nonzero-based and Cores-based. The best performance is obtained with OpenMP+MPI and the Core-based balance algorithm, which reaches a speedup of 36 with 112 cores. However, both OpenMP+MPI and Pthreads+MPI outperform UPC by large. The low system efficiency of 0.32 is mainly due to the slow internode communication network.