Can Your Context-Aware MT System Pass the DiP Benchmark Tests? : Evaluation Benchmarks for Discourse Phenomena in Machine Translation
Table 7: Discourse connective evaluation: Rankings of the different models for each language pair, ranked first by their Accuracy and then by the percentage where ANY connective is produced. Each set of rankings is followed by the results of the manual analysis on a subset of the translation data for Omissions, Synonyms, Mistranslations.
['De-En  [BOLD] Rank', 'De-En  [BOLD] Model', 'De-En  [BOLD] Acc', 'De-En  [BOLD] ANY', 'De-En  [BOLD] Om', 'De-En  [BOLD] Syn', 'De-En  [BOLD] Mis']
The ranking shows that Sen2Sen models are on average the most accurate and omit the connectives less often. Anaph continues its high performance in Ru-En, and while San leads the board for De-En in terms of accuracy, it has a low percentage of cases overall in which any connective is produced.