Tight Bounds for l1 Oblivious Subspace Embeddings

An l(p) oblivious subspace embedding is a distribution over r x n matrices. such that for any fixed n x d matrix A, Pr-Pi[for all x, ||Ax||(p) <= ||Pi Ax||(p) <= kappa||Ax||(p)] >= 9/10, where r is the dimension of the embedding,. is the distortion of the embedding, and for an n-dimensional vector y, ||y||(p) = (Sigma(n)(i=1) |y(i)|(p))(1/p) is the l(p)-norm. Another important property is the sparsity of Pi, that is, the maximum number of non-zero entries per column, as this determines the running time of computing Pi A. While for p = 2 there are nearly optimal tradeoffs in terms of the dimension, distortion, and sparsity, for the important case of 1 <= p < 2, much less was known. In this article, we obtain nearly optimal tradeoffs for l(1) oblivious subspace embeddings, as well as new tradeoffs for 1 < p < 2. Our main results are as follows: (1) We show for every 1 <= p < 2, any oblivious subspace embedding with dimension r has distortion kappa = Omega(1/(1/d)(1/p) log(2/p) r + (r/n)(1/p-1/2)). When r = poly(d) << n in applications, this gives a kappa = Omega(d(1/p) log(-2/p) d) lower bound, and shows the oblivious subspace embedding of Sohler and Woodruff (STOC, 2011) for p = 1 is optimal up to poly(log(d)) factors. (2) We give sparse oblivious subspace embeddings for every 1 <= p < 2. Importantly, for p = 1, we achieve r = O(d log d), kappa = O(d log d) and s = O(log d) non-zero entries per column. The best previous constructionwith s <= poly(log d) is due toWoodruff and Zhang (COLT, 2013), giving kappa = Omega(d(2)poly(log d)) or kappa = Omega(d(3/2) root log n center dot poly(log d)) and r >= d center dot poly(log d); in contrast our r = O(d log d) and kappa = O(d log d) are optimal up to poly(log(d)) factors even for dense matrices. We also give (1) l(p) oblivious subspace embeddings with an expected 1 + epsilon number of non-zero entries per column for arbitrarily small epsilon > 0, and (2) the first oblivious subspace embeddings for 1 <= p < 2 with O(1)-distortion and dimension independent of n. Oblivious subspace embeddings are crucial for distributed and streaming environments, as well as entrywise l(p) low-rank approximation. Our results give improved algorithms for these applications.