Promoting the Knowledge of Source Syntax in Transformer NMT Is Not Needed
Table 3: Comparison of BLEU scores at 600k training steps for linguistic and dummy secondary tasks with simple alternating approach.
['[BOLD] \xa0Model', '[BOLD] \xa0dev', '[BOLD] \xa0test']
“DepHeads” and “DepLabels” outperformed “CountSrcWords” and all other dummy tasks. This leads us to the conclusion that syntactic information is useful for the model, but the cost of the secondary task cancels all the gains. The yet a little worse result for “MT+DepHeads+DepLabels” confirms this: two additional tasks are more expensive than one.