Unsupervised and Interpretable Domain Adaptation to Rapidly Filter Social Web Data for Emergency Services
Table 6. Performance increases when additional web data is added for classifying TREC Priority task.
['TARGET', 'Single Task Acc', 'Single Task F1', 'Multi-Task (MTL) Acc', 'Multi-Task (MTL) F1', 'MTL+Amazon Acc', 'MTL+Amazon F1', 'MTL+Sentiment140 Acc', 'MTL+Sentiment140 F1']
In order to understand if additional web resources can further improve the performance of MT-DAAN, we expand it by adding new tasks from Amazon review dataset and Sentiment140 dataset. We select Priority as our primary classification task and design 5 more experiments: Single Task: Train solely on Priority. MTL: Multi-Task Learning where other classes such as Factoid, Sentiment, and Irrelevant are jointly trained. MTL+Amazon: Multi-Task Learning with an additional task to jointly train and classify Amazon reviews. MTL+Sentiment140: Multi-Task Learning with an additional task to jointly train and classify Sentiment140 positive/negative reviews. Both Amazon reviews and Sentiment140 datasets proved to be useful additions to the MTL setup showing that adding additional web resources may in fact help.