Encoding Source Language with Convolutional Neural Network for Machine Translation
Table 3: BLEU-4 scores (%) of inCNN models implemented with gating strategy and k max-pooling, where k is of {2, 4, 8}.
['[BOLD] Systems', '[BOLD] MT04', '[BOLD] MT05', '[BOLD] Average']
In this section, we investigate to what extent that our gating strategy can improve the translation performance over max pooling, with the comparisons on inCNN model as a case study. For implementation of inCNN with max-pooling, we replace the local-gating (Layer-2) with max-pooling with size 2 (2-pooling for short), and global gating (Layer-4) with k max-pooling (“k-pooling”), where k is of {2,4,8}. Then, we use the mean of the outputs of k-pooling as the final input of Layer-5. In doing so, we can guarantee the input dimension of Layer-5 is the same as the architecture with gating. Moreover, we find 8-pooling yields performance better than 2-pooling. We conjecture that this is because the useful relevant parts for translation are mainly concentrated on a few words of the source sentence, which can be better extracted with a larger pool size.