Adaptive iterative Hessian sketch via A-optimal subsampling

被引：5

作者：

Zhang, Aijun ^{[1
]}

Zhang, Hengtao ^{[1
]}

Yin, Guosheng ^{[1
]}

机构：

[1] Univ Hong Kong, Dept Stat & Actuarial Sci, Pokfulam Rd, Hong Kong, Peoples R China

来源：

STATISTICS AND COMPUTING | 2020年 / 30卷 / 04期

关键词：

Hessian sketch; Subsampling; Optimal design; Preconditioner; Exact line search; First-order method;

D O I：

10.1007/s11222-020-09936-8

中图分类号：

TP301 [理论、方法];

学科分类号：

081202 ;

摘要：

Iterative Hessian sketch (IHS) is an effective sketching method for modeling large-scale data. It was originally proposed by Pilanci and Wainwright (J Mach Learn Res 17(1):1842-1879, 2016) based on randomized sketching matrices. However, it is computationally intensive due to the iterative sketch process. In this paper, we analyze the IHS algorithm under the unconstrained least squares problem setting and then propose a deterministic approach for improving IHS via A-optimal subsampling. Our contributions are threefold: (1) a good initial estimator based on the A-optimal design is suggested; (2) a novel ridged preconditioner is developed for repeated sketching; and (3) an exact line search method is proposed for determining the optimal step length adaptively. Extensive experimental results demonstrate that our proposed A-optimal IHS algorithm outperforms the existing accelerated IHS methods.

引用

页码：1075 / 1090

页数：16

共 50 条

[21] Adaptive Optimal Iterative Learning Control for Local Ramp Metering
Jin, Shangtai
Hou, Zhongsheng
2008 WORKSHOP ON POWER ELECTRONICS AND INTELLIGENT TRANSPORTATION SYSTEM, PROCEEDINGS, 2008, : 122 - 126
[22] Optimal control of Hamiltonian systems via iterative learning
Fujimoto, K
SICE 2003 ANNUAL CONFERENCE, VOLS 1-3, 2003, : 2617 - 2622
[23] Iterative Optimal Preemphasis for Improved Glottal-Flow Estimation by Iterative Adaptive Inverse Filtering
Mokhtari, Parham
Ando, Hiroshi
18TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2017), VOLS 1-6: SITUATED INTERACTION, 2017, : 1044 - 1048
[24] Model-Free H∞ Optimal Tracking Control of Constrained Nonlinear Systems via an Iterative Adaptive Learning Algorithm
Hou, Jiaxu
Wang, Ding
Liu, Derong
Zhang, Yun
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS, 2020, 50 (11): : 4097 - 4108
[25] Research on Adaptive Optimal Iterative Learning Control Based on Data Driven
基于数据驱动的自适应最优迭代学习控制研究
Yang, Liangliang (yangliangliang@zstu.edu.cn); Yang, Liangliang (yangliangliang@zstu.edu.cn), 2021, Chinese Mechanical Engineering Society (57): : 207 - 216
[26] Adaptive iterative refinement of an optimal 2DF nonlinear controller
Keviczky, L
Báinyász, C
ADAPTATION AND LEARNING IN CONTROL AND SIGNAL PROCESSING 2001, 2002, : 425 - 430
[27] Optimal Coordination of Ramp Metering Via Iterative Dynamic Programming
Yu, X. F.
Xu, W. L.
Alam, F.
Fang, C. F.
INTERNATIONAL JOURNAL OF INTELLIGENT TRANSPORTATION SYSTEMS RESEARCH, 2015, 13 (03) : 203 - 218
[28] MISSING DATA RECOVERY VIA A NONPARAMETRIC ITERATIVE ADAPTIVE APPROACH
Stoica, Petre
Li, Jian
Ling, Jun
Cheng, Yubo
2009 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOLS 1- 8, PROCEEDINGS, 2009, : 3369 - +
[29] Missing Data Recovery Via a Nonparametric Iterative Adaptive Approach
Stoica, Petre
Li, Jian
Ling, Jun
IEEE SIGNAL PROCESSING LETTERS, 2009, 16 (04) : 241 - 244
[30] Network inference via adaptive optimal design
Stigter J.D.
Molenaar J.
BMC Research Notes, 5 (1)

← 1 2 3 4 5 →