All-in-one: Multi-task Learning for Rumour Verification
Table 5: Properties of the datasets for each of the events and each of the tasks, where S - Stance, V - Veracity, and D - Detection.
['[BOLD] Events', '[BOLD] Kurtosis  [BOLD] S', '[BOLD] Kurtosis  [BOLD] V', '[BOLD] Kurtosis  [BOLD] D', '[BOLD] Entropy  [BOLD] S', '[BOLD] Entropy  [BOLD] V', '[BOLD] Entropy  [BOLD] D', '[BOLD] TTR  [BOLD] S', '[BOLD] TTR  [BOLD] V', '[BOLD] TTR  [BOLD] D']
Alonso and Plank \shortcitealonso2017multitask link the gains/losses in performance from multi-task learning with information-theoretical metrics, properties of the label distributions: entropy (indicating the amount of uncertainty in the distribution) and kurtosis (indicating the skewness of the distribution). They have shown that the multi-task learning setup works best for tasks which have label distributions with lower kurtosis and relatively high entropy. Each of the events has very different properties; there is a strong difference in properties between larger events (top) and smaller ones (bottom). Smaller events tend to have more extreme values, which may explain that their addition to the evaluation adds further complexity to the task of rumour verification. In line with findings of Alonso and Plank \shortcitealonso2017multitask, in our case both auxiliary tasks have on average lower kurtosis than the main task. The stance classification dataset has on average higher entropy than the rumour detection dataset.