An Enhanced Apriori Algorithm Using Hybrid Data Layout Based on Hadoop for Big Data Processing

Frequent itemset mining is one of the data mining methodes implemeted to find frequent patterns, utilized in prediction, association rule mining, classification, etc. Apriori algorithm is an iterative method, that is used to discover frequent itemsets from transactional dataset. It scans entire dataset in every iteration to come up with the large frequent itemsets of various cardinality, which sounds efficient for small data but not useful for big data. To resolve the problem of treatment dataset in every iteration, we present an algorithm called Hybrid Frequent Itemset Mining on Hadoop (HFIMH) which uses the vertical layout of dataset to solve the problem of treatment the dataset in every iteration. Vertical dataset conveys information to discover support of every itemsets, and the idea of set intersection is utilized to compute it. We compare the execution of HFIMH with another Hadoop based implementation of Apriori algorithm for different datasets. Experimental results demonstrate that our approach is better.