Context-Gated Convolution
Table 6: Machine translation results on IWSLT’14 De-En. Param indicates the number of parameters in the model. Bold indicates the best result.
['Models', 'Param', 'BLEU-4']
Performance Comparisons. As is discussed previously, Dynamic Convolution leverages a linear layer to generate the convolution kernel according to the input segment, which lacks the awareness of global context. This flaw may lead to sub-optimal encoding of the source sentence and thus the unsatisfied decoded sentence. However, our CGC incorporates global context of the source sentence and helps significantly improve the quality of the translated sentence. Moreover, our CGC is much more efficient than Dynamic Convolution because of our module design. Our CGC only needs 0.01M extra parameters, but Dynamic Convolution needs 30× more.