SAFE: Unsupervised image feature extraction using self-attention based feature extraction network

被引:0
|
作者
Choi, Yeoung Je [1 ]
Lee, Gyeong Taek [2 ]
Kim, Chang Ouk [1 ,3 ]
机构
[1] Yonsei Univ, Dept Ind Engn, Seoul, South Korea
[2] Gachon Univ, Dept Mech Smart & Ind Engn, Seongnam, South Korea
[3] Yonsei Univ, Dept Ind Engn, Seoul 03722, South Korea
基金
新加坡国家研究基金会;
关键词
autoencoder; deep learning; feature representation; image processing; self-attention mechanism;
D O I
10.1111/exsy.13583
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The ability to extract high-quality features from data is critical for machine learning applications. With the development of deep learning, various methods have been developed for image feature extraction, and unsupervised techniques have gained popularity due to their ability to operate without response variables. Autoencoders with encoder-decoder architectures are a common example of such techniques, but they are limited by a lack of proportional relationship between model reconstruction and encoder feature extraction performance. If the decoder is composed of multiple layers and mapping to a higher dimension is easier, the feature extraction performance of the encoder is likely to decrease. However, previous research has not adequately addressed this limitation. This study identifies the limitations of conventional unsupervised feature extraction techniques that utilize the encoder-decoder architecture, and proposes a novel feature extraction technique called SAFE, which utilizes a self-attention mechanism to eliminate decoder effects and improve the performance of encoder. To validate the effectiveness of the proposed model, we conducted experiments using diverse datasets (MNIST, Fashion MNIST, SVHN, and WM811K). The results of the experiments demonstrated that our proposed method exhibited, on average, 2%-10% higher performance in terms of accuracy and F-measure compared to the existing feature extraction techniques in the classification problem. While our research has limitations, specifically in its applicability only to the selection of image features, future studies should be undertaken to explore its potential application in various fields.
引用
收藏
页数:16
相关论文
共 50 条
  • [11] Feature Extraction and Classification of Odor Using Attention Based Neural Network
    Fukuyama, Kohei
    Matsui, Kenji
    Omatsu, Sigeru
    Rivas, Alberto
    Manuel Corchado, Juan
    DISTRIBUTED COMPUTING AND ARTIFICIAL INTELLIGENCE, 16TH INTERNATIONAL CONFERENCE, 2020, 1003 : 142 - 149
  • [12] Synaptic Device Network Architecture with Feature Extraction for Unsupervised Image Classification
    Kim, Sungho
    Choi, Bongsik
    Lim, Meehyun
    Kim, Yeamin
    Kim, Hee-Dong
    Choi, Sung-Jin
    SMALL, 2018, 14 (32)
  • [13] Unsupervised EEG feature extraction based on echo state network
    Sun, Leilei
    Jin, Bo
    Yang, Haoyu
    Tong, Jianing
    Liu, Chuanren
    Xiong, Hui
    INFORMATION SCIENCES, 2019, 475 : 1 - 17
  • [14] An Unsupervised Feature Extraction Approach Based on Self-Expression
    Qu, Hongchun
    Zheng, Yangqi
    Li, Lin
    Guo, Fei
    BIG DATA, 2023, 11 (01) : 18 - 34
  • [15] A self-attention capsule feature pyramid network for water body extraction from remote sensing imagery
    Yu, Yongtao
    Yao, Yuting
    Guan, Haiyan
    Li, Dilong
    Liu, Zuojun
    Wang, Lanfang
    Yu, Changhui
    Xiao, Shaozhang
    Wang, Wenhao
    Chang, Lv
    INTERNATIONAL JOURNAL OF REMOTE SENSING, 2021, 42 (05) : 1801 - 1822
  • [16] Singing Melody Extraction Based on Combined Frequency-Temporal Attention and Attentional Feature Fusion with Self-Attention
    Qi, Xi
    Tian, Lihua
    Li, Chen
    Song, Hui
    Yan, Jiahui
    2022 IEEE INTERNATIONAL SYMPOSIUM ON MULTIMEDIA (ISM), 2022, : 220 - 227
  • [17] A COMPLEX NETWORK BASED FEATURE EXTRACTION FOR IMAGE RETRIEVAL
    Kang, Jieqi
    Lu, Shan
    Gong, Weibo
    Kelly, Patrick A.
    2014 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2014, : 2051 - 2055
  • [18] Multi scale pixel attention and feature extraction based neural network for image denoising
    Thakur, Ramesh Kumar
    Maji, Suman Kumar
    PATTERN RECOGNITION, 2023, 141
  • [19] Feature extraction and Cluster analysis of Pancreatic Pathological Image Based on Unsupervised Convolutional Neural Network
    Asano, Konosuke
    Ono, Naoaki
    Iwamoto, Chika
    Ohuchida, Kenoki
    Shindo, Koji
    Kanaya, Shigehiko
    PROCEEDINGS 2018 IEEE INTERNATIONAL CONFERENCE ON BIOINFORMATICS AND BIOMEDICINE (BIBM), 2018, : 2738 - 2740
  • [20] Self-attention Mechanism Network Integrating Spatio-Temporal Feature Extraction for Remaining Useful Life Prediction
    Zhang, Yiwei
    Liu, Kexin
    Zhang, Jiusi
    Huang, Lei
    JOURNAL OF ELECTRICAL ENGINEERING & TECHNOLOGY, 2025, 20 (02) : 1127 - 1142