Can Your Context-Aware MT System Pass the DiP Benchmark Tests? : Evaluation Benchmarks for Discourse Phenomena in Machine Translation
Table 11: Models ranked according to their performance (best to worst) in coherence according our evaluation, with BLEU for comparison. Coherence scores given here are obtained by subtracting the score for the model translation from the score for the reference translation, and summing the absolute score differences across the dataset. Hence, smaller model scores indicate better performance (closer to reference scores).
['Ru-En  [BOLD] Rank', 'Ru-En  [BOLD] BLEU', 'Ru-En  [BOLD] Coherence score']
The models are ranked based on these scores from lowest score (best performing) to highest score (worst performing).