Attention-based Encoder-Decoder Recurrent Neural Networks for HTTP Payload Anomaly Detection

Attack payloads are often short segments hidden in HTTP requests; thus they can be found by HTTP payload anomaly detection. Deep learning models learn data features during training without manual feature extraction, and better performance has received more attention. Recurrent Neural Network models process sequences directly, which are widely used in payload anomaly detection. However, due to the gradient vanishing problem, RNN has limits on processing the long sequences. Meanwhile, RNN uses its final hidden state for detection and pays more attention to the content of the end of the payload. Besides, deep learning generally lacks interpretability. The paper proposes an unsupervised deep learning model for HTTP payload Anomaly Detection, namely Attention-based Encoder-Decoder Recurrent Neural Networks Anomaly Detection model (AEDRAD). AEDRAD utilizes the encoder-decoder RNN and attention mechanism to detect anomalies by reconstructing the original sequences. AEDRAD filters the fields of HTTP protocol that cannot exist anomalies, focusing on the suspicious segments. Through the encoder-decoder network, the normal payload can be well-reconstructed while the anomaly payload fails. With the attention mechanism, AEDRAD generates practical features for further anomaly detection from a global perspective. Meanwhile, it marks abnormal fragments visually, which is conducive to a subsequent analysis by experts. The experimental results show that AEDRAD significantly outperforms three state-of-the-art unsupervised algorithms on two real datasets.