“Bilingual Expert” Can Find Translation Errors
Table 4: Results of word level QE on WMT 2017/2018
['[EMPTY]', 'F1-BAD', 'F1-OK', '[BOLD] F1-Multi']
The metric of word level is evaluated in terms of classification performance via the multiplication of F1-scores for the ‘OK’ and ‘BAD’ classes against the true labels. For the binary classification, we tuned the threshold of the classifier on the development data and applied to the test data. DCU is a stacked neural model by exploiting synergies between the related tasks of word-level quality estimation and automatic post-editing. In the primary metric F1-Multi, our algorithm of the single model outperforms all other models, including the best ensemble system in WMT17. In WMT18 word level QE task, our approach exceeds all other algorithms with significant better numbers.