Multi-branch Attentive TransformerThis work is conducted at Microsoft Research Asia.
Table 8: Results on code generation.
['Algorithm', 'Java BLEU', 'Java PoV', 'Python BLEU', 'Python PoV']
It is obvious that Transformer-based models (standard Transformer and MAT) is significantly better than the LSTM-based dual model. Compared with standard Transformer, our MAT can get 4.2 and 1.2 BLEU score improvement in Java and Python datasets respectively. MAT reaches the best BLEU scores when ρ=0.1 on both on Java and Python datasets, which shows that the drop branch technique is important in code generation task. When ρ=0.1, in terms of PoV, our MAT can boost the Transformer baseline by 5.8% and 6.6% points.