Explainable Emotion Classification in Social Media

Abstract

In the age of internet social media, people express their emotions openly online, making accurate emotion classification an increasingly important yet challenging task. Unlike sentiment analysis, which focuses on the polarity of opinions, emotion analysis dives deeper to identify specific emotions within text. Emotion classification has several practical applications, such as brand perception analysis, crisis management, and processing customer feedback, which are vital for businesses, governments, and organizations to engage effectively with their audiences. This thesis explores both multi-label and multi-class emotion classification and its explainability, specifically in tweets. For multi-label classification, we utilized the SemEval 2018 task E-c dataset, which comprises 11 distinct emotions, and for multi-class classification, we employed the DAIR AI emotion dataset. Through comprehensive experiments, we demonstrated robust model performance across both classification tasks, underscoring the adaptability and effectiveness of our approach. Our research uses instruction-based fine-tuning of large language models (LLMs) like GPT-2 and experiments with zero-shot and dynamic few-shot classification using GPT-4o, LLaMA 3 (8B), and DeepSeek R1 (Distilled Qwen 32B). Building on previous baseline performance, we further improved emotion detection and model interpretability by developing a self-explaining model that uses generative explanations and preference alignment. Notably, we constructed a novel preference alignment dataset using GPT-4o with chain-of-thought prompting, where human annotators assessed model outputs for correctness, clarity, helpfulness, and verbosity. Utilizing this dataset, we preference-aligned GPT-4o via Direct Preference Optimization (DPO) and open-source models, including LLaMA 3 (8B) and DeepSeek R1 (Distilled Qwen 32B), using Odds Ratio Preference Optimization (ORPO). The resulting self-explaining models achieved state-of-the-art multilabel classification performance (68.85\%) on the SemEval 2018 E-c dataset and competitive accuracy (93.1\%) on the DAIR AI multiclass dataset. Furthermore, the explanations generated by our models exceeded existing explainability techniques, achieving a higher sufficiency score of 63.66\%, reflecting better interpretability and alignment with human expectations. We also performed detailed human evaluations of the generated explanations, demonstrating that explanations produced by our preference-aligned models significantly surpass those from pre-trained models.