Evaluating Scoped Meaning Representations
Table 5: Comparing meaning representations of English texts to those of German, Italian and Dutch translations.
['[EMPTY]', '[BOLD] F-score%', '[BOLD] Docs', '[BOLD] F<1.0', '[BOLD] % total']
The second purpose of counter is checking whether translations are meaning-preserving. As a pilot study, we compare the gold standard meaning representations of German, Italian and Dutch translations in the release to their English counterparts. The high F-scores indicate that the meaning representations are often syntactically very similar, if not identical. However, there is a considerable subset of meaning representations which are different from the English ones, indicating that there is at least a slight discrepancy in meaning for those translations.