Causal Prediction of Top-k Event Types Over Real-Time Event Streams

This paper addresses the problem of causally predicting the top-k most likely next events over real-time event streams. Existing approaches have limitations-(i) they model causality in an acyclic causal network structure and search it to find the top-k next events, which does not work with real world event streams as they frequently manifest cyclic causality, and (ii) they prune out possible non-causal links from a causal network too aggressively and end up omitting many less frequent yet important causal links. We overcome these limitations using a novel event precedence model (EPM) and a run-time causal inference mechanism. The EPM constructs a Markov chain incrementally over event streams, where an edge between two events signifies a temporal precedence relationship between them, which is a necessary condition for causality. Then, the run-time causal inference mechanism performs causality tests on the EPM during query processing, and temporal precedence relationships that fail the causality test in the presence of other events are removed. Two query processing algorithms are presented. One performs exhaustive search on the model and the other performs more efficient reduced search with early termination. Experiments using two real data sets (cascading blackouts in power systems and web page views) verify efficacy and efficiency of the proposed probabilistic top-k prediction algorithms.