A Framework for Word Clustering of Bangla Sentences Using Higher Order N-gram Language Model

Clustering of words is the method that is used to partition the sets of words into subsets of semantically similar words. Word clustering has crucial in many uses of natural language processing like PUS tagging, spell checker, grammar checker, word sense disambiguation and many more. In this paper we propose a model by using higher order N-grams language model that is helpful for clustering Bangla word efficiently, which is based on the similarity of meaning in language and contextual. N-gram rules used to propagate various types of probabilities for different form of sentences. For implementation we also propose a system that generates different words of cluster and tested by threshold values to justify given result. By experimenting with a large corpus of the word length of Bangla sentences, our proposed model shows the accuracy approximately 89% for higher order N-gram which is quite satisfactory.