LogNADS: Network anomaly detection scheme based on log semantics representation

Semantics-aware anomaly detection based on log has attracted much attention. However, the existing methods based on the weighted aggregation of all word vectors might lose the semantic relationship of word order and cannot maintain the unique representation, and the methods based on word order-preserving by concatenating all word vectors might lead to a high computation time cost. To solve these issues and further improve the sequential anomaly detection, this paper proposes a network anomaly detection scheme LogNADS by designing a novel log semantics representation method and an adaptive sequence data construction method. It first discards the useless words and then selects theme words to hold the log abstraction and maintain a low time cost as well. Subsequently, it concatenates theme words' vectors based on the original word order to maintain the unique representation and avoid the word order loss. Furthermore, to better detect the sequential anomalies, we utilize the sliding window scheme and design a method to compute the optimal window size for constructing the log sequence self-adaptively, and then LSTM is built to extract timing characteristics of the log sequences. Experimental results conducted on the public benchmark HDFS dataset and BGL dataset demonstrate the effectiveness of LogNADS through comparing with other state-of-the-art methods in the detection accuracy and time cost. Moreover, the statistical significance tests prove the superior performance. (C) 2021 Elsevier B.V. All rights reserved.