Learning to Decode for Future Success
(c) Set with long targets.
['[EMPTY]', 'Q for MMI', 'MMI', 'SBS']
On the full test set, the future backward-probability prediction model outperforms the approach of reranking when decoding is fully finished. Specifically, a larger performance improvement is observed on examples with longer targets than on those with shorter ones. This effect is consistent with the intuition that for short responses, due to the relatively smaller search space, doing reranking at the end of decoding is sufficient, whereas this is not the case with longer sequences: as beam search proceeds, a small number of prefixes gradually start to dominate, with hypotheses differing only in punctuation or minor morphological variations. Incorporating mutual information in the early stages of decoding maintains diverse hypotheses, leading to better final results.