On estimating the rate-distortion function

Suppose a string X-1(n) = (X-1, X-2,..., X-n) is generated by a stationary memoryless source (X-n)(n >= 1) with unknown distribution P. When the source is finite-valued, the problem of estimating the entropy H(P) using the data X-1(n) has received a lot of attention. Perhaps the simplest method is the so-called plug-in estimator H(P-X1n), where P-X1n is the empirical distribution of the data X-1(n). This estimator is always strongly consistent, that is, H(P-X1n) -> H(P) with probability one, as n -> infinity. In this work we consider the natural generalization of estimating the rate-distortion function R(D, P). Our motivation comes from questions in lossy data compression and from cases where the data under consideration do not take values in a discrete alphabet. Our primary focus is the asymptotic behavior of the plug-in estimator R(P-X1n, D). This estimator need not be consistent, but in many cases it is. Several extensions are also considered, including stationary ergodic sources, and instances where the rate-distortion function is defined over a restricted class of coding distributions.