BOUNDS ON ACHIEVABLE CONVERGENCE-RATES OF PARAMETER ESTIMATORS VIA UNIVERSAL CODING

Lower bounds on achievable convergence rates of parameter estimators towards the true parameter are derived via universal coding considerations. It is shown that for a parametric class of finite-alphabet information sources, if there exists a universal lossless code whose redundancy decays sufficiently rapidly, then it induces a limitation on the fastest achievable convergence rate of any parameter estimator, at any value of the true paramete, with a possible exception of a vanishingly small subset of parameter values. A specific choice of a universal code yields a slightly different version of this result which extends easily to the continuous case.