Revisiting Round-Trip Translation for Quality Estimation
Table 2: Performance of RTT-based QE metrics on 22 English–German FT systems with various German–English BT systems. The variance of the best metrics, RTT-SBert and RTT-BERTScore, are described, additionally.
['Backward translations Systems', 'Backward translations Bleu', 'Pearson correlations RTT-Bleu', 'Pearson correlations RTT-chrF', 'Pearson correlations RTT-SBert', 'Pearson correlations RTT-BERTScore', 'Variance (×10−4) RTT-SBert', 'Variance (×10−4) RTT-BERTScore']
On the other hand, RTT-Bleu and RTT-chrF are sensitive to the performance of the BT systems, and the correlations fall behind RTT-BERTScore and RTT-SBert. Since BT systems scoring low Bleu have less chance of having same word orders in RTT as with input sentences, the performance of surface-form metrics, RTT-Bleu and RTT-chrF, decrease more sharply than RTT-SBert and RTT-BERTScore.