Synchronous Bidirectional Inference for Neural Sequence Generation
Table 2: Translation quality for Chinese-to-English tasks using case-insensitive BLEU scores. Our model BI-RNMT and BIFT are respectively significantly better than corresponding baselines (p < 0.01).
['Model', 'MT03', 'MT04', 'M05', 'MT06', 'AVE', 'Δ']
The results are mainly divided into two parts. The first part in this table shows the BLEU scores of the systems based on LSTM framework while the second part gives the results of various systems based on the Transformer architecture. As conventional decoding performs left to right, a question may arise that which inference direction is better. Specifically, the gap under the Transformer architecture is much bigger (1.08 vs. 0.29), indicating that Transformer is more sensitive to the inference direction. We also investigate previous methods that take advantage of two inference directions. The average gains can be up to 1.0 BLEU point (48.19 vs. 47.19), indicating that L2R decoding and R2L decoding can be complementary to each other. We go step further to exploit synchronous bidirectional inference that makes full use of L2R and R2L decoding. The second part demonstrates that our synchronous bidirectional inference model under the Transformer architecture BIFT achieves promising BLEU gains and the gap can be as large as 3.92 BLEU points on average. The remarkable improvements suggest that compared to asynchronous bidirectional decoding, our synchronous bidirectional inference can better explore the history and future contexts on the target side.