Can Your Context-Aware MT System Pass the DiP Benchmark Tests? : Evaluation Benchmarks for Discourse Phenomena in Machine Translation
Table 8: Results of the re-trained pronoun scoring model.
['[BOLD] Training data', '[BOLD] Test data', '[BOLD] Accuracy']
The pronoun evaluation model results reported in Jwalapuram et al. We re-train the model with more up-to-date data from WMT13-18, and test the model on WMT-19 data. Note that this training data is taken from WMT submissions, which do not overlap with the benchmarked MT models; there is therefore no conflict in using this trained model to evaluate the benchmarked model translations. Their model scores the translations in context; we provide the previous two sentences from the reference translation as context according to their settings.