A BAYESIAN INTERPRETATION OF THE LINEARLY-CONSTRAINED CROSS-ENTROPY MINIMIZATION PROBLEM

Both the linearly-constrained minimum cross-entropy (LCMXE) method and the Bayesian parameter estimation procedure have been widely used for solving various engineering problems. From the viewpoint of the information/decision theory, both approaches start with a prior distribution for a random variable, ''absorb'' new information, and finally produce a posterior distribution. In this paper, an equivalence relationship between these two approaches is established by identifying certain statistical experiments ''embedded'' in the LCMXE framework. Interestingly, the dual of the LCMXE problem actually ''translates'' the new information into its Bayesian counterpart. It is also shown that, while new information may come in stages, the identical final posterior can be obtained by applying the LCMXE method either stagewise or collectively. The equivalence further implies that the LCMXE method can help select a proper exponential family as the statistical model for the Bayesian experiments.