Boosting for high-multivariate responses in high-dimensional linear regression

We propose a boosting method, multivariate L-2 Boosting, for multivariate linear regression based on some squared error loss for multivariate data. It can be applied to multivariate linear regression with continuous responses and to vector autoregressive time series. We prove, for i.i.d. as well as time series data, that multivariate L-2 Boosting can consistently recover sparse high-dimensional multivariate linear functions, even when the number of predictor variables p(n) and the dimension of the response q(n) grow almost exponentially with sample size n, p(n) = q(n) = O(exp(Cn(1-xi))) (0 < xi < 1, 0 < C < infinity), but the l(1)-norm of the true underlying function is finite. Our theory seems to be among the first to address the issue of large dimension of the response variable; the relevance of such settings is briefly outlined. We also identify empirically some cases where our multivariate L-2 Boosting is better than multiple application of univariate methods to single response components, thus demonstrating that the multivariate approach can be very useful.