Learning to Create and Reuse Words in Open-Vocabulary Neural Language Modeling
Table 6: Results on MWC Corpus (bits-per-character).
['[EMPTY]', 'EN dev', 'EN test', 'FR dev', 'FR test', 'DE dev', 'DE test', 'ES dev', 'ES test', 'CS dev', 'CS test', 'FI dev', 'FI test', 'RU dev', 'RU test']
Similarly to WikiText-2 experiments, LSTM is strong baseline. We observe that the cache mechanism improve performance in every languages. In English, HCLM with cache achieved 1.538 bpc where the baseline is 1.622 bpc. It is 5.2% improvement. For other languages, the improvement rates were 2.7%, 3.2%, 3.7%, 2.5%, 4.7%, 2.7% in FR, DE, ES, CS, FI, RU respectively. The best improvement rate was obtained in Finnish.