Unsupervised Quality Estimation for Neural Machine Translation
Table 4: Pearson correlation (r) between sequence-level output probabilities (TP) and average DA for translations generated by different NMT systems.
['[BOLD] Method', '[ITALIC] r', '[BOLD] DA']
Interestingly, the correlation between output probabilities and DA is not necessarily related to the quality of MT outputs. For example, sampling produces much higher correlation although the quality is much lower. This is in line with previous work that indicates that sampling results in better calibrated probability distribution than beam search Ott et al. System variants that promote diversity in NMT outputs (diverse beam search and MoE) do not achieve any improvement in correlation over standard Transformer model.