Resolving Out-of-Vocabulary Words with Bilingual Embeddings in Machine Translation
Table 1: OOVs on the dev and test sets.
['[EMPTY]', 'Sent.', 'Tokens', 'OOVall', 'OOVCW']
The domainess of the test set is established with respect to the number of OOVs. Less than a 3% of the tokens are OOVs for News data (OOVall), whereas it is more than a 7% for Wikipedia ’s. In our experiments, we distinguish between OOVs that are named entities and the rest of content words (OOVCW). Only about 0.5% (NewsTest) and 1.8% (WikiTest) of the tokens fall into this category, but we show that they are relevant for the final performance. Next, we enrich the systemns with information gathered from the large monolingual corpora in two ways, using a bigger language model (BLM) and using our newly proposed log-bilinear model that uses word embeddings (BWE). BLMs are very important to improve the fluency of the translations, however they may not be helpful for resolving out-of-vocabulary words. On the other hand, BWEs are important to make available to the decoder new vocabulary on the topic of the otherwise OOVs.