Joint Learning of Pre-Trained and Random Units for Domain Adaptation in Part-of-Speech Tagging
Table 2: Comparison of our method to state-of-the-art (top) and baselines (bottom) in terms of token-level accuracy (in %) on 3 Tweets datasets. Note that, baselines are more fairly comparable to our method. In the second and last columns, we respectively highlighted the number of parameters and the average performance on the 3 datasets.
['[BOLD] Method', '[BOLD] #params', '[BOLD] TPoS  [ITALIC] Dev', '[BOLD] TPoS  [ITALIC] Test', '[BOLD] ArK  [ITALIC] Test', '[BOLD] TweeBank  [ITALIC] Dev', '[BOLD] TweeBank  [ITALIC] Test', '[BOLD] Avg']
More interestingly, PretRand significantly outperforms the popular fine-tuning baseline by +1.4% absolute point on average and is better on all classes (see per-class improvement on Fig. PretRand also outperforms the challenging Ensemble Model by a large margin (+2.2%), while using much less parameters. This clearly highlights the difference of our method with ensemble methods and the importance of having a shared word representation as well as our normalisation and weighting learnable vectors during training. A key asset of PretRand, is that it uses only 0.02% more parameters compared to the fine-tuning baseline.