On Optimal Data Compression in Multiterminal Statistical Inference

The multiterminal theory of statistical inference deals with the problem of estimating or testing the correlation of letters generated from two (or many) correlated information sources under the restriction of a certain transmission rate for each source. A typical example is two binary sources with joint probability p(x, y) where the correlation of x and y is to be tested or estimated. Given n iid observations x(n) = x(1) ... x(n) and y(n) = y(1) ... y(n), only k = rn (0 < r < 1) bits each can be transmitted to a common destination. What is the optimal data compression for statistical inference? A simple idea is to send the first k letters of x(n) and y(n). A simpler problem is the helper case where the optimal data compression of x(n) is searched for under the condition that all of y(n) are transmitted. It is a long standing problem to determine if there is a better data compression scheme than this simple scheme of sending first k letters. The present paper searches for the optimal data compression under the framework of linear-threshold encoding and shows that there is a better data compression scheme depending on the value of correlation. To this end, we evaluate the Fisher information in the class of linear-threshold compression schemes. It is also proved that the simple scheme is optimal when x and y are independent or their correlation is not too large.