Efficient Key Hash Indexing Scheme with Page Rank for Category Based Search Engine Big Data

Big Data, true to its name, it deals with large volumes of data characterized by volume, variety and velocity. Big data has made the development of highly capable online search engines nowadays. Search Engine systems are differing by the way of how Indexing and Page Rankings are performed. Without Indexing, Search Engine would require considerable time and computing power. For example, while an index of 10,000 documents can be queried within milliseconds, a sequential scan of every word in 10,000 large documents could take hours. The common indexing methods are not suitable for Search Engine Big Data as it greatly increase the size of the data as well as reduce the scalability. This paper proposes a new method of Indexing Search Engine Big Data called Key Hash Indexing scheme followed by the implementation of Page Rank. A Comprehensive presentation of important technology and factors to achieve efficient Big Data storage, Indexing and Ranking in Web Search Engine are also considered. This system also shows the efficiency of the method with an extensive set of experiments on real data. Experimental results on real-time data sets show that the proposed solution is effective as well as efficient in index generation and ranking.