Leveraging Declarative Knowledge in Text and First-Order Logic for Fine-Grained Propaganda Detection
Table 3: Detailed performance on the full task of fragment-level experiments (FLC task) on our test set. Precision (P), recall (R) and F1 scores per technique are provided.
['[BOLD] Propaganda Technique', '[BOLD] MGN  [BOLD] P', '[BOLD] MGN  [BOLD] R', '[BOLD] MGN F1', '[BOLD] LatexPRO  [BOLD] P', '[BOLD] LatexPRO  [BOLD] R', '[BOLD] LatexPRO F1', '[BOLD] LatexPRO (T+L)  [BOLD] P', '[BOLD] LatexPRO (T+L)  [BOLD] R', '[BOLD] LatexPRO (T+L) F1']
To further understand the performance of models for the FLC task, we make a more detailed analysis of each propaganda technique. Our models achieve precision and F1 improvements of almost all the classes over baseline model, and can also predict some low-frequency propaganda techniques, e.g., Whataboutism and Obfus.,Int. This further demonstrates that our method can stress class imbalance problem, and make more accurate predictions. To explore why our model LatexPRO We find that most of the off-diagonal elements are in class O which represents none of them. This demonstrates most of the cases are wrongly classified into O. We think this is due to the imbalance of the propaganda and non-propaganda categories in the dataset. Similarly, Straw Men, Red Herring and Whataboutism are the relatively low frequency of classes. How to deal with the class imbalance still needs further exploration.