Robust Multilingual Part-of-Speech Tagging via Adversarial Training
Table 4: POS tagging accuracy (test) on neighboring words. We cluster all words in the test set in the same way as Table 3 and consider the tagging performance on the neighbors (left and right) of these words in the test text.
['Word Frequency', '0', '1-10', '10-100', '100-', 'Total']
One important characteristic of natural language tasks is the sequential nature of inputs (i.e., sequence of words), where each word influences the function of its neighboring words. Since our model uses BiLSTM-CRF for that reason, we also study the tagging performance on the neighbors of rare / unseen words, and analyze the effects of AT with the sequence model in mind. We observe that AT tends to achieve large improvements over the baseline on the neighbors of unseen words (training frequency 0), while the improvements on the neighbors of more frequent words remain moderate. Our AT model thus exhibits strong stability to uncertain neighbors, as compared to the baseline. We suspect that because we generate adversarial examples against entire input sentences, training with adversarial examples makes the model more robust not only to perturbations in each word but also to perturbations in its neighboring words, leading to greater stability to uncertain neighbors.