Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks
Table 2: Improvements with transfer learning under multiple low-resource settings (%). “Dom”, “app”, and “ling” denote cross-domain, cross-application, and cross-lingual transfer settings respectively. The numbers following the slashes are labeling rates (chosen such that the number of labeled examples are of the same scale).
['Source', 'Target', 'Model', 'Setting', 'Transfer', 'No Transfer', 'Delta']
The numbers in the y-axes are accuracies for POS tagging, and chunk-level F1 scores for chunking and NER. We can see that our transfer learning approach consistently improved over the non-transfer results. We also observe that the improvement by transfer learning is more substantial when the labeling rate is lower. For cross-domain transfer, we obtained substantial improvement on the Genia and Twitter corpora by transferring the knowledge from PTB POS tagging and CoNLL 2003 NER. Cross-application transfer also leads to substantial improvement under low-resource conditions. We also study using different transfer learning models for the same task. We observe that the performance gain decreases when less parameters are shared (i.e., T-A > T-B > T-C).