Estimation of Error Variance in Regularized Regression Models via Adaptive Lasso

被引:4
|
作者
Wang, Xin [1 ]
Kong, Lingchen [1 ]
Wang, Liqun [2 ]
机构
[1] Beijing Jiaotong Univ, Dept Appl Math, Beijing 100044, Peoples R China
[2] Univ Manitoba, Dept Stat, Winnipeg, MB R3T 2N2, Canada
基金
中国国家自然科学基金;
关键词
high-dimensional linear model; variance estimation; natural adaptive lasso; mean squared error bound; regularized regression; NONCONCAVE PENALIZED LIKELIHOOD; VARIABLE SELECTION;
D O I
10.3390/math10111937
中图分类号
O1 [数学];
学科分类号
0701 ; 070101 ;
摘要
Estimation of error variance in a regression model is a fundamental problem in statistical modeling and inference. In high-dimensional linear models, variance estimation is a difficult problem, due to the issue of model selection. In this paper, we propose a novel approach for variance estimation that combines the reparameterization technique and the adaptive lasso, which is called the natural adaptive lasso. This method can, simultaneously, select and estimate the regression and variance parameters. Moreover, we show that the natural adaptive lasso, for regression parameters, is equivalent to the adaptive lasso. We establish the asymptotic properties of the natural adaptive lasso, for regression parameters, and derive the mean squared error bound for the variance estimator. Our theoretical results show that under appropriate regularity conditions, the natural adaptive lasso for error variance is closer to the so-called oracle estimator than some other existing methods. Finally, Monte Carlo simulations are presented, to demonstrate the superiority of the proposed method.
引用
收藏
页数:19
相关论文
共 50 条
  • [21] LASSO and shrinkage estimation in Weibull censored regression models
    Ahmed, S. Ejaz
    Hossain, Shakhawat
    Doksum, Kjell A.
    [J]. JOURNAL OF STATISTICAL PLANNING AND INFERENCE, 2012, 142 (06) : 1273 - 1284
  • [22] ADAPTIVE ESTIMATION OF REGRESSION-MODELS VIA MOMENT RESTRICTIONS
    NEWEY, WK
    [J]. JOURNAL OF ECONOMETRICS, 1988, 38 (03) : 301 - 339
  • [23] Manifold-Regularized Adaptive Lasso
    Chen, Si-Bao
    Zhang, Yu-Mei
    Luo, Bin
    [J]. ADVANCES IN BRAIN INSPIRED COGNITIVE SYSTEMS, BICS 2018, 2018, 10989 : 545 - 556
  • [24] Estimation and inference for error variance in bivariate nonparametric regression
    Bock, M.
    Bowman, A. W.
    Ismail, B.
    [J]. STATISTICS AND COMPUTING, 2007, 17 (01) : 39 - 47
  • [25] On Robust Estimation of Error Variance in (Highly) Robust Regression
    Kalina, Jan
    Tichavsky, Jan
    [J]. MEASUREMENT SCIENCE REVIEW, 2020, 20 (01): : 6 - 14
  • [26] Estimation and inference for error variance in bivariate nonparametric regression
    M. Bock
    A. W. Bowman
    B. Ismail
    [J]. Statistics and Computing, 2007, 17 : 39 - 47
  • [27] Greedy Variance Estimation for the LASSO
    Christopher Kennedy
    Rachel Ward
    [J]. Applied Mathematics & Optimization, 2020, 82 : 1161 - 1182
  • [28] Greedy Variance Estimation for the LASSO
    Kennedy, Christopher
    Ward, Rachel
    [J]. APPLIED MATHEMATICS AND OPTIMIZATION, 2020, 82 (03): : 1161 - 1182
  • [29] Conditional variance estimation in heteroscedastic regression models
    Chen, W-Hung
    Cheng, Ming-Yen
    Peng, Liang
    [J]. JOURNAL OF STATISTICAL PLANNING AND INFERENCE, 2009, 139 (02) : 236 - 245
  • [30] Variance estimation in high dimensional regression models
    Chatterjee, S
    Bose, A
    [J]. STATISTICA SINICA, 2000, 10 (02) : 497 - 515