Renyi Cross-Entropy Measures for Common Distributions and Processes with Memory

Two Renyi-type generalizations of the Shannon cross-entropy, the Renyi cross-entropy and the Natural Renyi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial networks. In this work, we derive the Renyi and Natural Renyi differential cross-entropy measures in closed form for a wide class of common continuous distributions belonging to the exponential family, and we tabulate the results for ease of reference. We also summarise the Renyi-type cross-entropy rates between stationary Gaussian processes and between finite-alphabet time-invariant Markov sources.