Predicting Target Language CCG Supertags Improves Neural Machine Translation
Table 3: Experiments with target-side syntax for German→English and Romanian→English. bleu scores reported for baseline NMT, syntax-aware NMT (SNMT) and multitasking. The SNMT system is also combined with source dependencies. Statistical significance is indicated with * p<0.05 and ** p<0.01, when comparing against the NMT baseline.
['model', 'syntax', 'strategy', 'German→English single', 'German→English ensemble', 'Romanian→English single', 'Romanian→English ensemble']
Target-side syntax We first evaluate the impact of target-side CCG supertags on overall translation quality. We report bleu scores for both the best single models and ensemble models. However, we will only refer to the results with ensemble models since these are generally better. Next, we evaluate whether there is a benefit to tight coupling between the target word sequence and syntax, as apposed to loose coupling. We compare our method of interleaving the CCG supertags with multitasking, which predicts target CCG supertags as a secondary task. For Romanian→English, which exhibits more local word reordering, multitasking improves bleu by 0.6 relative to the baseline. In contrast, the interleaving approach improves translation quality for both language pairs and to a larger extent. Therefore, we conclude that a tight integration of the target syntax and word sequence is important. Conditioning the prediction of words on their corresponding CCG supertags is what sets SNMT apart from the multitasking approach. Source-syntax may help more for Romanian→English because the training data is smaller and the word order is more similar between the source and target languages than it is for German→English.