Manifold Nearest Neighbor Sample Envelope and Hierarchical Multitype Transform Algorithm for Ensemble Learning

被引：0

作者：

Yan, Fang ^{[1
]}

Ma, Jie ^{[1
]}

Li, Yong-Ming ^{[1
]}

Wang, Pin ^{[1
]}

Qin, Jian ^{[1
]}

Liu, Cheng-Yu ^{[1
]}

机构：

[1] School of Microelectronics and Communication Engineering, Chongqing University, Chongqing,400044, China

来源：

Tien Tzu Hsueh Pao/Acta Electronica Sinica | 2024年 / 52卷 / 12期

关键词：

Ensemble learning is an important branch and research hotspot in machine learning. The current main paradigm of ensemble learning algorithms is to obtain multiple sample subsets based on the original sample set; then to train the base classifiers separately and integrate the base classifier results. The main problem of this paradigm is that the diversity among subsets is significantly reduced since all subsets are derived from the original sample set. This problem is especially serious when the data size of the original sample set is small; the sampling ratio is large; and the degree of imbalance is high. In addition; the improvement in the divisibility of the sample subsets obtained by resampling is also limited when the divisibility of the original sample set is low. In order to solve this problem; this paper proposes a manifold nearest neighbor sample envelope and hierarchical multitype transformation algorithm for ensemble learning. It aims to improve the diversity and divisibility of the sample subset by transforming the original sample set into a hierarchical enveloped sample set with differentiation through the envelopment mechanism and the multitype sample transformation. First; the manifold nearest neighbor sample envelope mechanism is designed to transform the original samples into sample envelopes. Second; a multitype sample transformation is performed on the sample envelope to reconstruct and generate hierarchical envelope samples. Third; the inter-layer consistency preservation mechanism based on joint structure domain adaptation is designed to preserve the distribution consistency of the samples before and after the transformation. Thus; improving the high representation ability of the envelope samples to the original samples. Four; feature dimensionality reduction and basic classifier training are performed separately for each layer of the envelope sample set. Finally; the final classification results are obtained using the two dimensional decision fusion mechanism. More than ten datasets and several representative algorithms are used in the experimental part for validation. The results show that compared with the original sample set; the proposed algorithm improves the diversity of the sample subsets; which improves the ensemble learning performance with up to 18.56% accuracy improvement. Compared with related ensemble learning algorithms; the accuracy of this paper’s algorithm has been improved by up to 7.56%. This paper provides a new idea for the improvement of existing ensemble learning algorithms; and it is valuable to transform the paradigm ofensemble learning directly based on original samplesinto a new paradigm ofensemble learning based on hierarchical envelope samples. © 2024 Chinese Institute of Electronics. All rights reserved;

D O I：

10.12263/DZXB.20231002

中图分类号：

学科分类号：

摘要：

引用

页码：4125 / 4141

共 50 条

[21] ANALOG CIRCUIT IMPLEMENTATION AND LEARNING ALGORITHM FOR NEAREST-NEIGHBOR CLASSIFIERS
URAHAMA, K
NAGAO, T
PATTERN RECOGNITION LETTERS, 1994, 15 (07) : 723 - 730
[22] A multi-relational hierarchical clustering algorithm based on shared nearest neighbor similarity
Guo, Jing-Feng
Zhao, Yu-Yan
Li, Jing
PROCEEDINGS OF 2007 INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND CYBERNETICS, VOLS 1-7, 2007, : 3951 - 3955
[23] A New K-nearest Neighbor Query Algorithm based on Grid Hierarchical Division
Li, Guobin
Tang, Jin'e
PROCEEDINGS OF 2010 3RD IEEE INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND INFORMATION TECHNOLOGY (ICCSIT 2010), VOL 6, 2010, : 255 - 257
[24] Rot-SiLA: A Novel Ensemble Classification approach based on Rotation Forest and Similarity Learning using Nearest Neighbor Algorithm
Shaheryar, Muhammad
Khalid, Mehrosh
Qamar, Ali Mustafa
2013 12TH INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND APPLICATIONS (ICMLA 2013), VOL 1, 2013, : 46 - 51
[25] Local Manifold Learning-Based k-Nearest-Neighbor for Hyperspectral Image Classification
Ma, Li
Crawford, Melba M.
Tian, Jinwen
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2010, 48 (11): : 4099 - 4109
[26] A novel hierarchical feature selection method based on large margin nearest neighbor learning
Zheng, Jian
Luo, Chuan
Li, Tianrui
Chen, Hongmei
NEUROCOMPUTING, 2022, 497 : 1 - 12
[27] Individual evolutionary algorithm and its application to learning of nearest neighbor based MLP
Zhao, QF
Higuchi, T
FROM NATURAL TO ARTIFICIAL NEURAL COMPUTATION, 1995, 930 : 396 - 403
[28] Ensemble Learning for Cluster Number Detection Based on Shared Nearest Neighbor Graph and Spectral Clustering
Zhang, Weihang
Ye, Xiucai
Sakurai, Testuya
2022 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2022,
[29] Motorcycle Apprehension using Deep Learning and K-Nearest Neighbor Algorithm
Garcia, Maria Rosario T.
Bandala, Argel A.
Dadios, Elmer P.
2021 6TH INTERNATIONAL CONFERENCE FOR CONVERGENCE IN TECHNOLOGY (I2CT), 2021,
[30] A hierarchical algorithm for approximate nearest neighbor searching in a dataset of pyramid-based image representations
Lange, M. M.
Lange, A. M.
3RD INTERNATIONAL CONFERENCE INFORMATION TECHNOLOGY AND NANOTECHNOLOGY (ITNT-2017), 2017, 201 : 302 - 311

← 1 2 3 4 5 →