Homotopy Based Algorithms for l0-Regularized Least-Squares

Sparse signal restoration is usually formulated as the minimization of a quadratic cost function vertical bar vertical bar y - Ax vertical bar vertical bar(2)(2) where A is a dictionary and x is an unknown sparse vector. It is well-known that imposing an l(0) constraint leads to an NP-hard minimization problem. The convex relaxation approach has received considerable attention, where the to-norm is replaced by the l(1)-norm. Among the many effective l(1) solvers, the homotopy algorithm minimizes vertical bar vertical bar y - Ax vertical bar vertical bar(2)(2) + lambda vertical bar vertical bar x vertical bar(1) with respect to x for a continuum of lambda's. It is inspired by the piecewise regularity of the l(1)-regularization path, also referred to as the homotopy path. In this paper, we address the minimization problem vertical bar vertical bar y - Ax vertical bar vertical bar(2)(2) + lambda vertical bar vertical bar x vertical bar(0) for a continuum of lambda's and propose two heuristic search algorithms for l(0)-homotopy. Continuation Single Best Replacement is a forward backward greedy strategy extending the Single Best Replacement algorithm, previously proposed for to-minimization at a given lambda. The adaptive search of the lambda-values is inspired by l(1)-homotopy. l(0) Regularization Path Descent is a more complex algorithm exploiting the structural properties of the l(0)-regularization path, which is piecewise constant with respect to lambda. Both algorithms are empirically evaluated for difficult inverse problems involving ill-conditioned dictionaries. Finally, we show that they can be easily coupled with usual methods of model order selection.