Semantic Neural Machine Translation using AMR
Table 3: Test performance. NC-v11 represents training only with the NC-v11 data, while Full means using the full training data. * represents significant Koehn (2004) result (p<0.01) over Seq2seq. ↓ indicates the lower the better.
['System', 'NC-v11 BLEU', 'NC-v11 TER↓', 'NC-v11 Meteor', 'Full BLEU', 'Full TER↓', 'Full Meteor']
Table 3 shows the TEST BLEU, TER and Meteor scores of all systems trained on the smallscale News Commentary v11 subset or the largescale full set. Dual2seq is consistently better than the other systems under all three metrics, [CONTINUE] Dual2seq is better than both OpenNMT-tf and Transformer-tf . [CONTINUE] Dual2seq is signifi [CONTINUE] cantly better than Seq2seq in both settings, [CONTINUE] In particular, the improvement is much larger under the small-scale setting (+3.2 BLEU) than that under the large-scale setting (+1.7 BLEU). [CONTINUE] When trained on the NC-v11 subset, the gap between Seq2seq and Dual2seq under Meteor (around 5 points) is greater than that under BLEU (around 3 points). [CONTINUE] As shown in the second group of Table 3, we further compare our model with other methods of leveraging syntactic or semantic information. Dual2seq-LinAMR shows much worse performance than our model and only slightly outperforms the Seq2seq baseline. [CONTINUE] Encoding dependency trees instead of AMRs, Dual2seq-Dep shows a larger performance gap with our model (17.8 vs 19.2) on small-scale training data than on large-scale training data (25.0 vs 25.5). [CONTINUE] Dual2seq-SRL is less effective than our model,