Morphosyntactic Tagging with a Meta-BiLSTM Modelover Context Sensitive Token Encodings
Table 8: F1 score of char models and their performance on the dev. set for selected languages with different gather strategies, concatenate to gi (Equation 1). DQM shows results for our reimplementation of DBLP:conf/conll/DozatQM17 (cf. §3.2), where we feed in only the characters. The final column shows the number of xpos tags in the training set.
['[BOLD] dev. set.  [BOLD] lang.', '[ITALIC] F [ITALIC] last  [ITALIC] B1 [ITALIC] st', '[ITALIC] F1 [ITALIC] st  [ITALIC] B [ITALIC] last', '[ITALIC] F [ITALIC] last  [ITALIC] B [ITALIC] last', '[ITALIC] F1 [ITALIC] st  [ITALIC] B1 [ITALIC] st', 'DQM', '|xpos|']
Thus, the proposed model concatenates all four of these and passes it as input to an multilayer perceptron (MLP): gi = concat(F1st(w),Flast(w), (1) B1st(w),Blast(w)) mcharsi = MLP(gi) A tag can then be predicted with a linear classifier that takes as input the output of the MLP mcharsi, applies a softmax function and chooses for each word the tag with highest probability. The Table also contains a column with results for our reimplementation of \newciteDBLP:conf/conll/DozatQM17. We removed, for all systems, the word model in order to assess each strategy in isolation. The performance is quite different per language. E.g., for Latin, the outputs of the forward and backward LSTMs of the last character scored highest.