On NMT Search Errors and Model Errors: Cat Got Your Tongue?
Table 1: NMT with exact inference. In the absence of search errors, NMT often prefers the empty translation, causing a dramatic drop in length ratio and BLEU.
['[BOLD] Search', '[BOLD] BLEU', '[BOLD] Ratio', '[BOLD] #Search errors', '[BOLD] #Empty']
Our main result is shown in Tab. > token. Fig. Large beam sizes reduce the number of search errors, but the BLEU score drops because translations are too short. Even a large beam size of 100 produces 53.62% search errors. Fig. For example, Beam-10 yields 15.9% fewer search errors (absolute) than greedy decoding (57.68% vs. 73.58%), but Beam-100 improves search only slightly (53.62% search errors) despite being 10 times slower than beam-10.