Research on hidden Markov model for system call anomaly detection

Intrusion detection, especially anomaly detection, requires sufficient security background knowledge. It is very significant to recognize system anomaly behavior under the condition of poor domain knowledge. In this paper, the general methods for system calls anomaly detection are summarized and HMM used for anomaly detection is deeply discussed from detection theory, system framework and detection methods. Moreover, combining with experiments, the detection efficiency and real-time performance of HMM with all-states transition and part-states transition are analyzed in detail in the paper.