Resolving Out-of-Vocabulary Words with Bilingual Embeddings in Machine Translation
Table 2: Automatic evaluation of the translation systems defined in Section 2. The best system is bold-faced (see text for statistical significance).
['[EMPTY]', 'NewsTest TER', 'NewsTest BLEU', 'NewsTest MTR', 'WikiTest TER', 'WikiTest BLEU', 'WikiTest MTR']
We consider two baseline systems, the first one does not output any translation for OOVs (noOOV), it just ignores the token; the second one outputs a verbatim copy of the unseen word as a translation (verbatimOOV). Including the verbatim copy improves all the lexical evaluation metrics. Specially for named entities and acronyms (the 80% of OOVs in our sets), this is a hard baseline to beat since in most cases the same word is the correct translation (e.g. Messi, PHP, Sputnik…).