Multilingual Part-of-Speech Tagging withBidirectional Long Short-Term Memory Models and Auxiliary Loss
Table 2: Tagging accuracies on UD 1.2 test sets. →w: words, →c: characters, →b: bytes. Bold/†: best accuracy/representation; +Polyglot: using pre-trained embeddings. Freqbin: our multi-task model. OOV Acc: accuracies on OOVs. BTS: best results in gillick:ea:2016 (not strictly comparable).
['[EMPTY]', 'Baselines Tnt', 'Baselines Crf', 'bi-LSTM using: → [ITALIC] w', 'bi-LSTM using: → [ITALIC] c', 'bi-LSTM using: → [ITALIC] c+→ [ITALIC] b', 'bi-LSTM using: → [ITALIC] w+→ [ITALIC] c', '→ [ITALIC] w+→ [ITALIC] c +Polyglot bi-LSTM', '→ [ITALIC] w+→ [ITALIC] c +Polyglot Freqbin', 'OOV Acc bi-LSTM', 'OOV Acc Freqbin', 'BTS']
First of all, notice that TnT performs remarkably well across the 22 languages, closely followed by CRF. The bi-LSTM tagger (→w) without lower-level bi-LSTM for subtokens falls short, outperforms the traditional taggers only on 3 languages. The bi-LSTM model clearly benefits from character representations. The model using characters alone (→c) works remarkably well, it improves over TnT on 9 languages (incl. Slavic and Nordic languages). The combined word+character representation model is the best representation, outperforming the baseline on all except one language (Indonesian), providing strong results already without pre-trained embeddings. This model (→w+→c) reaches the biggest improvement (more than +2% accuracy) on Hebrew and Slovene. Initializing the word embeddings (+Polyglot) with off-the-shelf language-specific embeddings further improves accuracy. The only system we are aware of that evaluates on UD is \newcitegillick:ea:2016 (last column). However, note that these results are not strictly comparable as they use the earlier UD v1.1 version. The overall best system is the multi-task bi-LSTM freqbin (it uses →w+→c and Polyglot initialization for →w). While on macro average it is on par with bi-LSTM →w+→c, it obtains the best results on 12/22 languages, and it is successful in predicting POS for OOV tokens (cf.