Unbabel’s Participationin the WMT19 Translation Quality Estimation Shared Task
Table 4: Results of document-level submissions, and their performance of the dev and dev0 validation sets.
['[EMPTY]', 'Dev', 'Dev0', 'Test']
On F1 annotations, results across all data sets are reasonably consistent. On the other hand, MQM Pearson varies significantly between dev and dev0. Differences in the training of the two systems shouldn’t explain this variation, since both have equivalent performance on the test set.