The Role of Pragmatic and Discourse Contextin Determining Argument Impact
Table 6: F1 scores of each model for the claims with various context length values.
['[EMPTY]', 'C [ITALIC] l=1', 'C [ITALIC] l=2', 'C [ITALIC] l=3', 'C [ITALIC] l=4']
To understand for what kinds of claims the best performing contextual model is more effective, we evaluate the BERT model with flat context representation for claims with context length values 1, 2, 3 and 4 separately. For the claims with context length 1, adding Contextf(3) and Contextf(4) representation along with the claim achieves significantly better (p<0.05) F1 score than modeling the claim only. Similarly for the claims with context length 3 and 4, Contextf(4) and Contextf(3) perform significantly better than BERT with claim only ((p<0.05) and (p<0.01) respectively). We see that models with larger context are helpful even for claims which have limited context (e.g. Cl=1). This may suggest that when we train the models with larger context, they learn how to represent the claims and their context better.