Layer-Wise Cross-View Decoding for Sequence-to-Sequence Learning
Table 2: Experimental results of machine translation in terms of BLEU. † denotes statistically significant results (t-test with p<0.01). As a whole, the proposed cross-view decoding with granularity consistent attention significantly improves the baselines.
['Methods', 'EN-DE', 'EN-FR', 'DE-EN']
Results We report results on three benchmarks, including two large WMT-2014 datasets, i.e., English-German (EN-DE) and English-French (EN-FR), and a small IWSLT-2014 dataset, i.e., German-English (DE-EN). For three datasets, our approach outperforms all the baselines. Based on the Transformer, we promote the baseline by 0.9, 1.5 and 1.2 BLEU score for the EN-DE, EN-FR and DE-EN, respectively. More encouragingly, based on the DynamicConv, which is the previous state-of-the-art, our approach sets a new state-of-the-art performance on three datasets, achieving 29.8, 43.5 and 36.2 BLEU score on EN-DE, EN-FR and DE-EN respectively. The improvements on various datasets demonstrate the effectiveness of the proposal. Since GCA simply reroutes the original source information without conducting further transformations, it also suggests that the expressive power of the existing models are overlooked and providing the decoder with the representation of the appropriate granularity has great potential in improving the parameter efficiency.