Large Scale Online Kernel Learning

被引:0
|
作者
Lu, Jing [1 ]
Hoi, Steven C. H. [1 ]
Wang, Jialei [2 ]
Zhao, Peilin [3 ]
Liu, Zhi-Yong [4 ]
机构
[1] Singapore Management Univ, Sch Informat Syst, 80 Stamford Rd, Singapore 178902, Singapore
[2] Univ Chicago, Dept Comp Sci, 5050 S Lake Shore Dr Apt S2009, Chicago, IL 60637 USA
[3] ASTAR, Inst Infocomm Res, 1 Fusionopolis Way,21-01 Connexis, Singapore 138632, Singapore
[4] Chinese Acad Sci, State Key Lab Management & Control Complex Syst, 95 Zhongguancun East Rd, Beijing 100190, Peoples R China
关键词
online learning; kernel approximation; large scale machine learning;
D O I
暂无
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In this paper, we present a new framework for large scale online kernel learning, making kernel methods efficient and scalable for large-scale online learning applications. Unlike the regular budget online kernel learning scheme that usually uses some budget maintenance strategies to bound the number of support vectors, our framework explores a completely different approach of kernel functional approximation techniques to make the subsequent online learning task efficient and scalable. Specifically, we present two different online kernel machine learning algorithms: (i) Fourier Online Gradient Descent (FOGD) algorithm that applies the random Fourier features for approximating kernel functions; and (ii) Nystrom Online Gradient Descent (NOGD) algorithm that applies the Nystrom method to approximate large kernel matrices. We explore these two approaches to tackle three online learning tasks: binary classification, multi-class classification, and regression. The encouraging results of our experiments on large-scale datasets validate the effectiveness and efficiency of the proposed algorithms, making them potentially more practical than the family of existing budget online kernel learning approaches.
引用
收藏
页数:43
相关论文
共 50 条
  • [21] Kernel-Based Autoencoders for Large-Scale Representation Learning
    Bao, Jinzhou
    Zhao, Bo
    Guo, Ping
    2021 7TH INTERNATIONAL CONFERENCE ON ROBOTICS AND ARTIFICIAL INTELLIGENCE, ICRAI 2021, 2021, : 112 - 117
  • [22] Sparse Approximation Through Boosting for Learning Large Scale Kernel Machines
    Sun, Ping
    Yao, Xin
    IEEE TRANSACTIONS ON NEURAL NETWORKS, 2010, 21 (06): : 883 - 894
  • [23] Approximate kernel extreme learning machine for large scale data classification
    Iosifidis, Alexandros
    Tefas, Anastasios
    Pitas, Ioannis
    NEUROCOMPUTING, 2017, 219 : 210 - 220
  • [24] FPGA Fastfood - A High Speed Systolic Implementation of a Large Scale Online Kernel Method
    Fox, Sean
    Boland, David
    Leong, Philip
    PROCEEDINGS OF THE 2018 ACM/SIGDA INTERNATIONAL SYMPOSIUM ON FIELD-PROGRAMMABLE GATE ARRAYS (FPGA'18), 2018, : 279 - 284
  • [25] Large Scale Online Multiple Kernel Regression with Application to Time-Series Prediction
    Sahoo, Doyen
    Hoi, Steven C. H.
    Li, Bin
    ACM TRANSACTIONS ON KNOWLEDGE DISCOVERY FROM DATA, 2019, 13 (01)
  • [26] Large Scale Online Learning of Image Similarity through Ranking
    Chechik, Gal
    Sharma, Varun
    Shalit, Uri
    Bengio, Samy
    PATTERN RECOGNITION AND IMAGE ANALYSIS, PROCEEDINGS, 2009, 5524 : 11 - +
  • [27] Approximation Vector Machines for Large-scale Online Learning
    Trung Le
    Tu Dinh Nguyen
    Vu Nguyen
    Dinh Phung
    JOURNAL OF MACHINE LEARNING RESEARCH, 2017, 18
  • [28] A Empirical Study of Programming Behaviours on Large Scale Online Learning
    Cao, Menghua
    Zhang, Qian
    14TH INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND EDUCATION (ICCSE 2019), 2019, : 889 - 894
  • [29] Understanding Student Disaffection in Large-Scale Online Learning
    Hew, Khe Foon
    15TH IEEE INTERNATIONAL CONFERENCE ON ADVANCED LEARNING TECHNOLOGIES (ICALT 2015), 2015, : 17 - 18
  • [30] Voted Spheres: An Online, Fast Approach to Large Scale Learning
    Farran, Bassam
    Saunders, Craig
    2009 INTERNATIONAL CONFERENCE ON ADVANCED INFORMATION NETWORKING AND APPLICATIONS WORKSHOPS: WAINA, VOLS 1 AND 2, 2009, : 744 - 749