Fine-Tune Longformer for Jointly Predicting Rumor Stance and Veracity
Table 4. Test results for Task B
['[BOLD] Rank', '[BOLD] System', '[BOLD] MacroF', '[BOLD] RMSE']
In this section we evaluate our proposed approach against the top-5 systems submitted at RumorEval 2019(Gorrell We have conducted the separate experiments, to properly investigate the performance of a) each of the classifiers used in the ensemble. b) impact of NLP features on each of those classifiers and finally, c) the performance of our proposed system. at the fine-tuned post representation as shown in figure. Similarly, Longformer + Identity Encoder + NLP Features, Longformer + Transformer + NLP Features, Longformer + BiLSTM + NLP Features are corresponding models with NLP features. Specifically, for the model named Longformer + Identity Encoder is only a base model (Longformer) without any encoder to judge the performance boost as compared to when we use BiLSTM or Transformer encoder. Top- Each model is chosen randomly and if it increases the ensemble’s F1 then it has been added to the ensemble. To create the pool of models, we consider only models trained with NLP features since they have better performance, i.e. D, E & F. Finally, Our proposed method represents the model averaging of models based on three architectures(D, E & F) with NLP Features trained with varying parameters (encoder and learning rate). In this paper, we have evaluated our models using the same guidelines as in RumorEval 2019 contest Specifically, they have used macro-averaged F1 to evaluate the performance on both Task A and B. Additionally, they have used RMSE score for Task B to judge the confidence scores. paper(Gorrell it is clear that our proposed method shows the best performance among all the approaches. These results also state the importance of NLP Features and Sentence Encoder. We will discuss the effect of each in the following sections.