Annotation of a Corpus of Tweets for Sentiment Analysis

This article describes the process of creation and annotation of a tweets corpus for Sentiment Analysis at sentence level. The tweets were captured using the #masterchefbr hashtag, in a tool to acquire the public stream of tweets in real time and then annotated based on the six basic emotions (joy, surprise, fear, sadness, disgust, anger) commonly used in the literature. The neutral tag was adopted to annotate sentences where there was no expressed emotion. At the end of the process, the measure of disagreement between annotators reached a Kappa value of 0.42. Some experiments with the SVM algorithm (Support Vector Machine) have been performed with the objective of submitting the annotated corpus to a classification process, to better understand the Kappa value of the corpus. An accuracy of 52.9% has been obtained in the classification process when using both discordant and concordant text within the corpus.