Detecting Ideological Framing in Geopolitical News: Fine-Tuning RoBERTa-Large with Algorithmic Resampling for Binary Bias Classification
Automated identification of political frame effects in biased news texts continues to be a difficult NLP problem, motivated by issues of label distribution imbalance and contextual richness. Despite being successful at traditional text classification tasks, identification of complex geopolitical bias effects becomes hindered by imbalanced datasets with an overrepresentation of the majority class. In this paper, we propose a binary classification pipeline for identification of geopolitical framing biases within a collection of 30,000 short news articles in the AG News dataset "World" sub-category. In order to overcome imbalances in the minority class, we used stratified partitioning and random sampling techniques for achieving class balance in a training set with 43,698 examples. We fine-tuned RoBERTa-Large (355M parameters) model for three epochs with sequence classification task using FP16 precision strategy. The resulting classifier reaches 91.08% accuracy (95% CI: [90.46%, 91.66%], Wilson score method), macro-F1 equal to 0.91 and AUC score of 0.9641 (95% CI: [0.9601, 0.9681], Hanley-McNeil approximation). Pair-wise McNemar's significance test shows that RoBERTa-Large model significantly outperforms ALBERT-base (χ2 = 5.68, p = 0.017) and DistilBERT (χ2 = 155.42, p < 0.0001) transformers which were independently fine-tuned using the same strategy. Experimental findings show that data balancing combined with transformer representations provide a statistically sound solution for bias identification.