A Note on Entropy Estimation

We compare an entropy estimator ($) over cap (z) recently discussed by Zhang (2012) with two estimators, ($) over cap (1) and ($) over cap (2), introduced by Grassberger (2003) and Schurmann (2004). We prove the identity ($) over cap (z) equivalent to ($) over cap (1), which has not been taken into account by Zhang (2012). Then we prove that the systematic error (bias) of ($) over cap (1) is less than or equal to the bias of the ordinary likelihood (or plug-in) estimator of entropy. Finally, by numerical simulation, we verify that for the most interesting regime of small sample estimation and large event spaces, the estimator ($) over cap (2) has a significantly smaller statistical error than ($) over cap (z).