Transfer Learning from Transformers to Fake News Challenge Stance Detection (FNC-1) Task
Table 5: Performance of featMLP model without and with InferSent- and BERT- based features
['Score, %', 'Base', 'Inf1', 'Inf3', 'BERT1', 'BERT3', 'BERT +Inf3', 'BERT only']
These three features together slightly improve F1 macro and F1 -”agree”, -”disagree” ” Inf1” column reports the impact of InferSent embeddings for input sequence pair, ”Inf3” shows the impact of this feature together with two similarity scores for two sequences in the input pair. BERT1” column shows the impact of BERT vector representation of an input, and ”BERT3” shows the impact of this feature together with two similarity scores between BERT embeddings of two sequences in the input pair. The column ”BERT3 + Inf3” reports InferSent and BERT cumulative effect. The most significant F1-score increase, that of 7% is attained for most difficult ”disagree” class.