Fusion k-means clustering and multi-head self-attention mechanism for a multivariate time prediction model with feature selection

As the demand for precise predictions grows across various industries due to advancements in sensor technology and computer hardware, multi-feature time series prediction shows significant promise in fields such as information fusion, finance, energy, and meteorology. However, traditional machine learning methods often struggle to forecast future events given the increasing complexity of the data. To address this challenge, the paper introduces an innovative approach that combines an improved k-means clustering with a multi-head self-attention mechanism. This method utilizes long and short-term memory (LSTM) neural networks to filter and identify the most effective feature subset for prediction. In the enhanced k-means clustering algorithm, a novel similarity formula named Feature Vector Similarity (FVS) and a method for automatically determining the number of clustering centers are proposed. This advancement improves the rationality of cluster center selection and enhances overall clustering performance. The multi-head self-attention mechanism calculates the clustering centers and attention weights of objects within the cluster partitions, optimizing feature selection and enhancing computational efficiency. The fusion of k-means clustering, the multi-head self-attention mechanism, and LSTM networks results in a new feature selection method, referred to as KMAL. To further refine the prediction process, we integrate KMAL with LSTM, known for its strong performance in predicting long-term time series, to develop a novel prediction model: KMAL-LSTM. In the subsequent comparative experiments, the prediction performance of the models is assessed using mean absolute error (MAE), mean bias error (MBE), and root mean square error (RMSE). The proposed KMAL-LSTM model consistently exhibits superior validity, stability, and performance when compared to seven other prediction models across six distinct datasets.