Multi-task Additive Models for Robust Estimation and Automatic Structure Discovery

被引:0
|
作者
Wang, Yingjie [1 ]
Chen, Hong [2 ]
Zheng, Feng [3 ]
Xu, Chen [4 ]
Gong, Tieliang [4 ,5 ]
Chen, Yanhong [6 ]
机构
[1] Huazhong Agr Univ, Coll Informat, Wuhan, Peoples R China
[2] Huazhong Agr Univ, Coll Sci, Wuhan, Peoples R China
[3] Southern Univ Sci & Technol, Dept Comp Sci & Engn, Shenzhen, Peoples R China
[4] Univ Ottawa, Dept Math & Stat, Ottawa, ON, Canada
[5] Xi An Jiao Tong Univ, Sch Comp Sci & Technol, Xian, Peoples R China
[6] Chinese Acad Sci, Natl Space Sci Ctr, Beijing, Peoples R China
来源
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 33, NEURIPS 2020 | 2020年 / 33卷
基金
加拿大自然科学与工程研究理事会; 中国国家自然科学基金;
关键词
MINIMAX-OPTIMAL RATES; REGRESSION;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Additive models have attracted much attention for high-dimensional regression estimation and variable selection. However, the existing models are usually limited to the single-task learning framework under the mean squared error (MSE) criterion, where the utilization of variable structure depends heavily on a priori knowledge among variables. For high-dimensional observations in real environment, e.g., Coronal Mass Ejections (CMEs) data, the learning performance of previous methods may be degraded seriously due to the complex non-Gaussian noise and the insufficiency of a prior knowledge on variable structure. To tackle this problem, we propose a new class of additive models, called Multi-task Additive Models (MAM), by integrating the mode-induced metric, the structure-based regularizer, and additive hypothesis spaces into a bilevel optimization framework. Our approach does not require any priori knowledge of variable structure and suits for high-dimensional data with complex noise, e.g., skewed noise, heavy-tailed noise, and outliers. A smooth iterative optimization algorithm with convergence guarantees is provided to implement MAM efficiently. Experiments on simulations and the CMEs analysis demonstrate the competitive performance of our approach for robust estimation and automatic structure discovery.
引用
收藏
页数:12
相关论文
共 50 条
  • [1] Additive multi-task learning models and task diagnostics
    Miller, Nikolay
    Zhang, Guoyi
    COMMUNICATIONS IN STATISTICS-SIMULATION AND COMPUTATION, 2024, 53 (12) : 6120 - 6137
  • [2] Benchmarking multi-task learning in predictive models for drug discovery
    McCabe, Philippa Grace
    Ortega-Martorell, Sandra
    Olier, Ivan
    2019 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2019,
  • [3] AutoDistiller: An Automatic Compression Method for Multi-task Language Models
    Wang, Hongsheng
    Xiao, Geyang
    Liang, Yuan
    2022 41ST CHINESE CONTROL CONFERENCE (CCC), 2022, : 2410 - 2415
  • [4] Robust partially linear models for automatic structure discovery
    Han, Yuxiang
    Chen, Hong
    Gong, Tieliang
    Cai, Jia
    Deng, Hao
    EXPERT SYSTEMS WITH APPLICATIONS, 2023, 217
  • [5] Multi-task ordinal regression with task weight discovery
    Xiao, Yanshan
    Zeng, Mengyue
    Liu, Bo
    Zhao, Liang
    Kong, Xiangjun
    Hao, Zhifeng
    KNOWLEDGE-BASED SYSTEMS, 2024, 305
  • [6] ADAPTIVE AND ROBUST MULTI-TASK LEARNING
    Duan, Yaqi
    Wang, Kaizheng
    ANNALS OF STATISTICS, 2023, 51 (05): : 2015 - 2039
  • [7] Group LASSO with Asymmetric Structure Estimation for Multi-Task Learning
    Oliveira, Saullo H. G.
    Goncalves, Andre R.
    Von Zuben, Fernando J.
    PROCEEDINGS OF THE TWENTY-EIGHTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2019, : 3202 - 3208
  • [8] Towards Automatic Construction of Multi-Network Models for Heterogeneous Multi-Task Learning
    Garciarena, Unai
    Mendiburu, Alexander
    Santana, Roberto
    ACM TRANSACTIONS ON KNOWLEDGE DISCOVERY FROM DATA, 2021, 15 (02)
  • [9] Multi-task Sparse Structure Learning with Gaussian Copula Models
    Goncalves, Andre R.
    Von Zuben, Fernando J.
    Banerjee, Arindam
    JOURNAL OF MACHINE LEARNING RESEARCH, 2016, 17 : 1 - 30
  • [10] Robust multi-task distributed estimation based on generalized maximum correntropy criterion
    胡倩
    陈枫
    叶明
    Chinese Physics B, 2023, (06) : 812 - 822