Shannon's differential entropy asymptotic analysis in a Bayesian problem

We consider a Bayesian problem of estimating of probability of success in a series of conditionally independent trials with binary outcomes. We study the asymptotic behaviour of the differential entropy for a posterior probability density function conditional on x successes after n conditionally independent trials, when n --> infinity. Three particular cases are studied: x is a proportion of n; x similar to n(beta), where 0 < beta < 1; either x or n - x is a constant. It is shown that after an appropriate normalization in the first and second case limiting distribution is Gaussian and the differential entropy of a standardized RV converges to the differential entropy of a the Gaussian distribution. In the third case, the limiting distribution in not Gaussian, but still the asymptotics of the differential entropy can be found explicitly.