The Edit Distance Transducer in Action: The University of Cambridge English-German System at WMT16
Table 3: Breakdown of the distances measured between NMT and Hiero along the shortest path in C on news-test2016.
['[BOLD] Distance measure component', '[BOLD] Avg.\xa0number per sentence', '[BOLD] Percentage of affected sentences']
Next, we investigate the distance measure between NMT and Hiero translations, which is realized with the UNK insertion transducer U and the modified edit distance transducer E (Sec. Tab. The average edit distance between NMT and Hiero disregarding UNKs on the best path (type III) is 1.74. In 61.7% of the cases the input and output labels differ not only at UNK – i.e. in only 38.3% of the sentences do we have an exact match between NMT and Hiero. We note that UNK is often replaced with an NMT in-vocabulary word (55.9% of the sentences). It seems that NMT often produces an UNK even if a better word is in the NMT vocabulary. This could be due to the over-representation of UNK in the NMT training corpus.