Sentiment Analysis on Short Social Media Texts Using DistilBERT

Abstract

Sentiment analysis on short texts from social media, such as tweets, presents unique challenges due to their brevity and informal language. This study explores the effectiveness of transformer-based models, particularly DistilBERT, in performing sentiment analysis on short texts compared to traditional machine learning approaches including Support Vector Machine, Logistic Regression, and Naive Bayes. The objective is to assess whether DistilBERT not only enhances sentiment classification accuracy but also remains efficient enough for quick social media analysis. The models used in this study were trained and evaluated on stratified samples of 10,000, 30,000, and 50,000 tweets, drawn from the Sentiment140 dataset while preserving the original class distribution. The methodology involved data collection and sampling, data splitting, data cleaning, feature extraction, model training, and evaluation using accuracy and F1-score. Experimental results showed that DistilBERT consistently outperformed traditional models in both accuracy and F1-score, and demonstrated competitive results against BERT while requiring significantly less training time. Specifically, DistilBERT trained approximately 1.8 times faster than BERT on average, highlighting its computational efficiency. The best result was achieved by DistilBERT trained on the 50k subset, reaching an accuracy of 85% and an F1-score of 84%. These findings suggest that lightweight transformer models like DistilBERT are highly suitable for real-world sentiment analysis tasks where both speed and performance are critical.