LLM-based multi-dimensional analysis outperforms RoBERTa sentiment models in political news evaluation

A new arXiv preprint introduces a comparative study evaluating RoBERTa-based sentiment analysis against an LLM-based multi-dimensional framing analysis platform. The research applies both methods to a corpus of 50 political news articles sourced from 17 international media outlets.

The authors identify a phenomenon they term ‘neutral collapse,’ where RoBERTa classifies 70% of the articles as neutral. This outcome flattens substantively rich political content into an analytically uninformative category, limiting the utility of traditional sentiment analysis for political discourse research.

The study further reports that 23% of the articles labeled neutral by RoBERTa exhibit negative probability scores above 0.30, indicating underlying negative sentiment that is obscured by the neutral classification. This discrepancy underscores the limitations of binary or coarse-grained sentiment models in capturing nuanced political expression.

By contrast, the LLM-based approach demonstrates the ability to capture multiple dimensions of political news, including political bias direction and intensity, sensationalism, emotional appeal, and political framing. These outputs are described as more aligned with the epistemological needs of social sciences and humanities (SSH) research.

The paper argues that traditional sentiment analysis alone is insufficient for political media analysis and advocates for the adoption of LLM-based multi-dimensional frameworks to better address the complex analytical requirements of SSH research.