Why Self-Attention?A Targeted Evaluation of Neural Machine Translation Architectures
Table 3: The performance of ConvS2S with different settings. K means the kernel size. The ctx column is the theoretical largest local context size in the masked decoder.
['Layer', 'K', 'Ctx', '2014', '2017', 'Acc(%)']
In order to explore how the ability of ConvS2S to capture long-range dependencies depends on the local context size, we train additional systems, varying the number of layers and kernel size. The results indicate that the accuracy increases when the local context size becomes larger, but the BLEU score does not. Moreover, ConvS2S is still not as good as RNNS2S for subject-verb agreement.