Efficient mining of long frequent patterns from very large dense datasets

Discovering association rules that identify relationships among sets of items in a transaction database is an important problem in Data Mining. Finding frequent itemsets has been an active research area since it is the crucial step in association rule discovery. However, efficiently mining frequent itemsets from dense datasets is still a challenging problem. In this paper, we describe a new and more efficient algorithm named CT-GIN for mining complete frequent itemsets from dense datasets. The algorithm uses a compact prefix tree for succinctly representing transaction data and an item group intersection method for efficient extraction of frequent itemsets from the tree. Performance comparisons show that our algorithm outperforms the fastest Apriori algorithm, Eclat and FP-Growth, on several widely used test data sets. CT-GIN has been extended for mining very large datasets, and we also present test results showing its scalability.