On Sentiment of Online Fake News

The presence of disinformation and fake news on the Internet and especially social media has become a major concern. Prime examples of such fake news surged in the 2016 U.S. presidential election cycle and the COVID-19 pandemic. We quantify sentiment differences between true and fake news on social media using a diverse body of datasets from the literature that contains about 100K previously labeled true and fake news. We also experiment with a variety of sentiment analysis tools. We model the association between sentiment and veracity as conditional probability and also leverage statistical hypothesis testing to uncover the relationship between sentiment and veracity. With a significance level of 99.999%, we observe a statistically significant relationship between negative sentiment and fake news and between positive sentiment and true news. The degree of association, as measured by Goodman and Kruskal's gamma, ranges between .037 to .475. Finally, we make our data and code publicly available to support reproducibility. Our results assist in the development of automatic fake news detectors.