Multi-channel Encoder for Neural Machine Translation
Table 1: Performance of different systems on the NIST Chinese-to-English translation tasks. Compared to the strong open source system DL4MT, our models achieve significant improvements. We also list results from another two recently published open source toolkits T2T and ConvS2S for comparison. Remember that both T2T and ConvS2S are deep models with multiple layers.
['SYSTEMS', 'PARAMETERS', 'NIST03', 'NIST04', 'NIST05', 'NIST08', 'NIST12', 'AVG']
To confirm that our system is strong enough, we also report performance of an open source system: DL4MT. At first, we find that there is a significant improvements of our systems over the DL4MT. Compared to the DL4MT, our basic RNN system achieves an improvement by 4.94 BLEU points. Although our RNN is a basic attention-based NMT, we assemble it with some advanced techniques, such as initialization all parameters uniformly, adding biases for the embedding vectors, using the output of forward RNN as the input of the backward RNN and training with dynamic learning rate. Whatever, we give the comparison between our basic RNN system and open source toolkits is to prove that our baseline is strong enough, and all improvements over baseline system are reliable.