Analyzing Mismatch Between Human and LLM-Predicted Hashtags: A Sentiment-Based Evaluation Using LIWC and VADER

This study applies semantic and sentiment analysis to explain why large language model (LLM)–predicted hashtags differ from hashtags chosen by human content creators for YouTube long-form video descriptions. Using the Public Health Advocacy Dataset (PHAD), which contains social-media videos related to tobacco products (University of Arkansas CVIU Lab, n.d.), the project examines whether the sentiment expressed in each description particularly emotional tone or motivational language, helps explain why some LLM predictions match human labels and others do not. An LLM (Qwen-3) predicts hashtags based solely on video descriptions, and mismatches between predicted and human-assigned hashtags are then analyzed. In this study, two approaches are used to measure sentimental features: LIWC categories capturing tones, and curiosity-related wording, and VADER polarity scores catching fine-grained emotional tone. Both sentiment models are applied to the validation dataset to compare matched and mismatched cases. The LLM reached an accuracy of 55.19%. Results show no significant sentiment differences between correct and incorrect predictions, suggesting that mismatches are not driven by emotional or motivational cues and that the LLM’s errors are more likely related to semantic ambiguity or category complexity rather than sentiment.