Non-autoregressive Machine Translation with Disentangled Context Transformer
Table 7: Dev results with different decoding strategies.
['[BOLD] Inference Strategy', '[BOLD] en\\rightarrowde Step', '[BOLD] en\\rightarrowde BLEU', '[BOLD] ro\\rightarrowen Step', '[BOLD] ro\\rightarrowen BLEU']
Alternative Inference Algorithms ’s probability conditioned on the easier positions from the previous iteration. We evaluate two alternative orderings: left-to-right and right-to-left. We see that both of them yield much degraded performance. We also attempt to use even broader context than parallel easy-first by computing the probability at each position based on all other positions (all-but-itself, Y_{obs}^{n,t}=Y_{\neq n}^{t-1}). We again see degraded performance, suggesting that cyclic dependency (e.g., Y_{m}^{t-1}\in Y_{{\tt obs}}^{n,t} and Y_{n}^{t-1}\in Y_{{\tt obs}}^{m,t}) breaks consistency. For example, a model can have two output candidates: “Hong Kong” and “New York” (Zhang2020POINTERCT). In this case, we might end up producing “Hong York” due to this cyclic dependency. These results suggest that the easy-first ordering we introduced is a simple yet effective approach.