Opinion Mining based complex polarity shift pattern handling for improved sentiment classification

The availability of rich volume of information on the web makes sentiment analysis an interesting, informative and impressive task. There is now an increasing desire to mine this information to gain new insights. But how well we extract value from this wealth of information is much important because identifying, accurately analyzing and extracting subjective information from the available text is not an easy task. Hence the purpose of this research work is to effectively handle complex polarity shift patterns such as negations, intensifiers, diminishers and contrast transitions that affect (sometimes, reverse) the polarity of the text for better classification of sentiments and nine classification techniques in supervised learning were used for binary sentiment classification. The most commonly used standard labelled dataset with movie reviews was used. The implementation was done in Python using the libraries of Natural Language Toolkit (NLTK) and Scikit-Learn (sklearn). Grid Search cross validation technique was also used to choose best parameters for each model while training so that the classifier could accurately predict the testing data. The accuracy of sentiment classification task of the original dataset was compared with the one after complex polarity shift pattern handling. The results of this study show an increase in the accuracy up to 3.5% for various classifiers after polarity shift pattern handling. Random Forest classifier which is an ensemble of classifiers tops the list with high accuracy among the other classifiers used.