Sumset Inequalities for Differential Entropy and Mutual Information

The Plunnecke-Ruzsa sumset theory gives bounds connecting the cardinality of the sumset A + B defined as {alpha + b; a is an element of A, b is an element of B} with the cardinalities of the original sets A, B. For example, the sum-difference bound states that, vertical bar A + B vertical bar vertical bar A vertical bar vertical bar B vertical bar <= vertical bar A - B vertical bar(3), where A - B = {alpha - b; a is an element of A, b is an element of B}. Interpreting the differential entropy h(X) as (the logarithm of) the size of the effective support of X, the main results here are a series of natural information-theoretic analogs for these bounds. For example, the sum-difference bound becomes the new inequality, h(X + Y) + h(X) + h(Y) <= 3h(X - Y), for independent X, Y. Our results include differential-entropy versions of Ruzsa's triangle inequality, the Plunnecke-Ruzsa inequality, and the Balog-Szemeredi-Gowers lemma. Versions of most of these results for the discrete entropy H(X) were recently proved by Tao, relying heavily on a strong, functional form of the submodularity property of H(X). Since differential entropy is not functionally submodular, in the continuous case many of the corresponding discrete proofs fail, in several cases requiring substantially new proof strategies. The basic property that naturally replaces functional submodularity is the data processing property of mutual information.