Manifold Nearest Neighbor Sample Envelope and Hierarchical Multitype Transform Algorithm for Ensemble Learning

被引：0

作者：

Yan, Fang ^{[1
]}

Ma, Jie ^{[1
]}

Li, Yong-Ming ^{[1
]}

Wang, Pin ^{[1
]}

Qin, Jian ^{[1
]}

Liu, Cheng-Yu ^{[1
]}

机构：

[1] School of Microelectronics and Communication Engineering, Chongqing University, Chongqing,400044, China

来源：

Tien Tzu Hsueh Pao/Acta Electronica Sinica | 2024年 / 52卷 / 12期

关键词：

Ensemble learning is an important branch and research hotspot in machine learning. The current main paradigm of ensemble learning algorithms is to obtain multiple sample subsets based on the original sample set; then to train the base classifiers separately and integrate the base classifier results. The main problem of this paradigm is that the diversity among subsets is significantly reduced since all subsets are derived from the original sample set. This problem is especially serious when the data size of the original sample set is small; the sampling ratio is large; and the degree of imbalance is high. In addition; the improvement in the divisibility of the sample subsets obtained by resampling is also limited when the divisibility of the original sample set is low. In order to solve this problem; this paper proposes a manifold nearest neighbor sample envelope and hierarchical multitype transformation algorithm for ensemble learning. It aims to improve the diversity and divisibility of the sample subset by transforming the original sample set into a hierarchical enveloped sample set with differentiation through the envelopment mechanism and the multitype sample transformation. First; the manifold nearest neighbor sample envelope mechanism is designed to transform the original samples into sample envelopes. Second; a multitype sample transformation is performed on the sample envelope to reconstruct and generate hierarchical envelope samples. Third; the inter-layer consistency preservation mechanism based on joint structure domain adaptation is designed to preserve the distribution consistency of the samples before and after the transformation. Thus; improving the high representation ability of the envelope samples to the original samples. Four; feature dimensionality reduction and basic classifier training are performed separately for each layer of the envelope sample set. Finally; the final classification results are obtained using the two dimensional decision fusion mechanism. More than ten datasets and several representative algorithms are used in the experimental part for validation. The results show that compared with the original sample set; the proposed algorithm improves the diversity of the sample subsets; which improves the ensemble learning performance with up to 18.56% accuracy improvement. Compared with related ensemble learning algorithms; the accuracy of this paper’s algorithm has been improved by up to 7.56%. This paper provides a new idea for the improvement of existing ensemble learning algorithms; and it is valuable to transform the paradigm ofensemble learning directly based on original samplesinto a new paradigm ofensemble learning based on hierarchical envelope samples. © 2024 Chinese Institute of Electronics. All rights reserved;

D O I：

10.12263/DZXB.20231002

中图分类号：

学科分类号：

摘要：

引用

页码：4125 / 4141

共 50 条

[31] An Orthogonal Wavelet Transform-Based K-Nearest Neighbor Algorithm to Detect Faults in Bearings
Li, Weipeng
Cao, Yan
Li, Lijuan
Hou, Siyu
SHOCK AND VIBRATION, 2022, 2022
[32] A QUASI-NEWTON ALGORITHM ON THE ORTHOGONAL MANIFOLD FOR NMF WITH TRANSFORM LEARNING
Ablin, Pierre
Fagot, Dylan
Wendt, Herwig
Gramfort, Alexandre
Fevotte, Cedric
2019 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2019, : 700 - 704
[33] Equal-average equal-variance nearest neighbor search algorithm based on hadamard transform
Lu, ZM
Pei, H
2003 INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND CYBERNETICS, VOLS 1-5, PROCEEDINGS, 2003, : 2976 - 2978
[34] Quantum K-Nearest-Neighbor Image Classification Algorithm Based on K-L Transform
Zhou, Nan-Run
Liu, Xiu-Xun
Chen, Yu-Ling
Du, Ni-Suo
INTERNATIONAL JOURNAL OF THEORETICAL PHYSICS, 2021, 60 (03) : 1209 - 1224
[35] Quantum K-Nearest-Neighbor Image Classification Algorithm Based on K-L Transform
Nan-Run Zhou
Xiu-Xun Liu
Yu-Ling Chen
Ni-Suo Du
International Journal of Theoretical Physics, 2021, 60 : 1209 - 1224
[36] Human Activity Recognition Using K-Nearest Neighbor Machine Learning Algorithm
Mohsen, Saeed
Elkaseer, Ahmed
Scholz, Steffen G.
SUSTAINABLE DESIGN AND MANUFACTURING, KES-SDM 2021, 2022, 262 : 304 - 313
[37] Breast Cancer Detection using K-nearest Neighbor Machine Learning Algorithm
Al-hadidi, Mohd Rasoul
Alarabeyyat, Abdulsalam
Alhanahnah, Mohannad
2016 9TH INTERNATIONAL CONFERENCE ON DEVELOPMENTS IN ESYSTEMS ENGINEERING (DESE 2016), 2016, : 35 - 39
[38] Context Awareness System on Ubiquitous Learning with Case Based Reasoning and Nearest Neighbor Algorithm
Selviandro, Nungki
Sabariah, Mira Kania
Saputra, Surya
2016 4TH INTERNATIONAL CONFERENCE ON INFORMATION AND COMMUNICATION TECHNOLOGY (ICOICT), 2016,
[39] State prediction of algae reproduction based on learning algorithm of fuzzy nearest neighbor clustering
Zhang, Ying
PROGRESS IN ENVIRONMENTAL SCIENCE AND ENGINEERING (ICEESD2011), PTS 1-5, 2012, 356-360 : 929 - 932
[40] Sample-Pair Envelope Diamond Autoencoder Ensemble Algorithm for Chronic Disease Recognition
Zhang, Yi
Ma, Jie
Qin, Xiaolin
Li, Yongming
Zhang, Zuwei
APPLIED SCIENCES-BASEL, 2023, 13 (12):

← 1 2 3 4 5 →