Optimizing Sparse Tensor Times Matrix on Multi-core and Many-core Architectures

This paper presents the optimized design and implementation of sparse tensor-times-dense matrix multiply (SpTTM) for CPU and GPU platforms. This primitive is a critical bottleneck in data analysis and mining applications based on tensor methods, such as the Tucker decomposition. We first design and implement sequential SpTTM to avoid explicit data transformations between a tensor and a matrix, which is the conventional approach. We further optimize SpTTM on multicore CPU and GPU systems by parallelizing, avoiding locks, and exploiting data locality. Our sequential SpTTM is up to 3.5x faster than the SpTTM from Tensor Toolbox and 1.5x over that from Cyclops Tensor Framework. Our parallel algorithms show 4.1x speedup on multicore Intel Core i7 and 18.8x speedup on NVIDIA K40c GPU over our sequential SpTTM respectively.