Recursive Renyi's Entropy estimator for adaptive filtering

Recently we have proposed a recursive estimator for Renyi's quadratic entropy. This estimator can converge to accurate results for stationary signals or track the changing entropy of nonstationary signals. In this paper, we demonstrate the application of the recursive entropy estimator to supervised and unsupervised training of linear and nonlinear adaptive systems. The simulations suggest a smooth and fast convergence to the optimal solution with a reduced complexity in the algorithm compared to a batch training approach using the same entropy-based criteria. The presented approach also allows on-line information theoretic adaptation of model parameters.