Linguistic evaluation of German-English Machine Translation using a Test Suite
Tabelle 2: Percentage (%) of accuracy improvement or deterioration between WMT18 and WMT19 for all the systems submitted (averaged in last column) and the systems submitted with the same name
['category', '#', 'JHU', 'MLLP', 'onlA', 'onlB', 'onlG', 'onlY', 'RWTH', 'UCAM', 'UEDIN', 'avg']
Although this is not entirely accurate, since different systems participate, we assume that the large amount of the test items allows some generalisations to this direction. When one compares the overall accuracy, there has been an improvement of about 6%. When focusing on particular categories, the biggest improvements are seen at function words (+12.5%), non-verbal agreement (+9.7%) and punctuation (+8%). The smallest improvement is seen at named entity and terminology (+0.3%).