Tree-to-Sequence Attentional Neural Machine Translation
Table 3: Evaluation results on the development data using the small training data. The training time per epoch is also shown, and K is the number of negative samples in BlackOut.
['[EMPTY]', '[ITALIC] K', 'Perplexity', 'RIBES', 'BLEU', 'Time/epoch (min.)']
We conducted the experiments with our proposed method using BlackOut and softmax. We decoded a translation by our proposed beam search with a beam size of 20. Although the result of softmax is better than those of BlackOut (K=500,2000), the training time of softmax per epoch is about three times longer than that of BlackOut even with the small dataset. One of the possible reasons is that BlackOut distorts the target word distribution by the modified unigram-based negative sampling where frequent words can be treated as the negative samples multiple times at each training step.