Comparative analysis of machine learning models for solar flare prediction

In this paper, we develop five machine learning models, neural network (NN), long short-term memory (LSTM), LSTM based on attention mechanism (LSTM-A), bidirectional LSTM (BLSTM), and BLSTM based on attention mechanism (BLSTM-A), for predicting whether a ≥C class or ≥M class flare will occur in an active region in the next 24 hr. We use the data base provided by the Space-weather Helioseismic and Magnetic Imager Active Region Patches of Solar Dynamic Observatory, including 10 magnetic field features of active regions from 2010 May 1 to 2018 September 13. The samples are labeled flare information (i.e. No-flare/C/M/X) using solar flare events catalogue provided by the Geostationary Operational Environmental Satellite and Solar Geophysical Data solar event reports. In addition, we generated 10 cross-validation sets from these data using the cross-validation method. Then, after training, validating, and testing our models, we compare the results with the true skill statistics (TSS) as the assessment metric. The main results are as follows. (1) The TSS scores for ≥C class are 0.5472 ± 0.0809, 0.6425 ± 0.0685, 0.6904 ± 0.0575, 0.6681 ± 0.0573, and 0.6833 ± 0.0531 for NN, LSTM, LSTM-A, BLSTM and BLSTM-A, respectively. The TSS scores for ≥M class are 0.5723 ± 0.1139, 0.6579 ± 0.0758, 0.5943 ± 0.0712, 0.6493 ± 0.0826, and 0.5932 ± 0.0723, respectively. (2) For the first time, we add an attention mechanism to BLSTM for flare prediction, which improves the performance of the model for ≥C class. (3) Among the five models, the prediction model based on deep learning algorithms is generally superior to the model based on the traditional machine learning algorithm. The performance of the LSTM models is comparable to that of the BLSTM models. In general, LSTM-A for ≥C class performs better than other models. In addition, we also discuss the influence of 10 features on LSTM-A, and we find that removing the least significant feature will result in better performance than using all 10 features together, and the TSS score of the model will improve to 0.7059 ± 0.0440.