Better Character Language Modeling Through Morphology
Table 6: Results on Multilingual Wikipedia Corpus (MWC) in bits per character (BPC), trained on the vocabulary from Mielke and Eisner (2019).
['[EMPTY]', '[BOLD] CS dev', '[BOLD] CS test', '[BOLD] DE dev', '[BOLD] DE test', '[BOLD] EN dev', '[BOLD] EN test', '[BOLD] ES dev', '[BOLD] ES test', '[BOLD] FI dev', '[BOLD] FI test', '[BOLD] FR dev', '[BOLD] FR test', '[BOLD] RU dev', '[BOLD] RU test']
The character vocabulary provided in the MWC dataset and used for the distant supervision setting differs from the vocabulary calculated by including the characters that occur more than 25 times in the MWC training set. We find that our LM and MTL models generally obtain slightly better performance on this setting, and we continue to see improvement from multitasking morphology over the character LM baseline.