Density Estimation by Total Variation Penalized Likelihood Driven by the Sparsity l1 Information Criterion

We propose a non-linear density estimator, which is locally adaptive, like wavelet estimators, and positive everywhere, without a log- or root-transform. This estimator is based on maximizing a non-parametric log-likelihood function regularized by a total variation penalty. The smoothness is driven by a single penalty parameter, and to avoid cross-validation, we derive an information criterion based on the idea of universal penalty. The penalized log-likelihood maximization is reformulated as an l(1)-penalized strictly convex programme whose unique solution is the density estimate. A Newton-type method cannot be applied to calculate the estimate because the l(1)-penalty is non-differentiable. Instead, we use a dual block coordinate relaxation method that exploits the problem structure. By comparing with kernel, spline and taut string estimators on a Monte Carlo simulation, and by investigating the sensitivity to ties on two real data sets, we observe that the new estimator achieves good L-1 and L-2 risk for densities with sharp features, and behaves well with ties.