Neural Language Model Based Training Data Augmentation for Weakly Supervised Early Rumor Detection
TABLE IV: Comparison of the paraphrase identification performance of different models for sentence representation.
['[BOLD] Model', '[BOLD] F', '[BOLD] P', '[BOLD] R', '[BOLD] Threshold']
We are interested in exploring the effect of the distance between embeddings of pairs of reference and candidate tweets on the quality of augmented data which will eventually affect the rumor detection model’s capability to predict unseen rumors. We show the results based on the maximum F-score each model achieved. Our experimental results show the effectiveness of our CREDBANK fine-tuned ELMo over pre-trained model (”Original (5.5B)”) and SoA word embedding models. We applied different models to normalized texts. In fact, state-of-the-art NLMs like ELMo do not need much text normalization. A pre-trained ELMo model only needs tokenization. As for the output of ELMo models, using the average of representations from all layers outperforms using only the top layer representation. This finding is consistent with results presented in Perone et al. To ensure higher quality (i.e., less false positives in a selected sample), relatedness thresholds are fine-tuned based on precision achieved by the best-performing model. We should choose a threshold which can achieve a reasonably high precision and sample an adequate number of tweets.