HIGH DIMENSIONAL EXPONENTIAL FAMILY ESTIMATION VIA EMPIRICAL BAYES

Consider estimating theta(1),..., theta(N) ON based on data z(i) f(theta i), where f(theta) is a continuous natural exponential family. One successful approach is Empirical Bayes (EB). EB methods assume that theta come from an unknown prior, and estimate the Bayes procedure corresponding to that prior. In this paper, we propose a general form of nonparametric EB estimator that uses estimates of the marginal density of z and its derivative. This estimator was first proposed by Zhang (1997) for the normal means problem, z(i) similar to N (theta(i), 1). We bound the regret of our method in terms of the error in estimating the marginal density and its derivative. As a side point, our proof yields a lower bound on the regret of general estimators. We illustrate our method in the simultaneous chi-squared problem, where z is a chi-squared random variable with scale 1/theta(i). We specialize our theoretical results to this case, and study the empirical performance of our method under different estimators of the marginal and its derivative. Our method outperforms the UMVU estimator and a conjugate prior parametric EB approach.