Lattice Transformer for Speech Translation
Table 6: BLEU on WMT 2017 Chinese English
['Architecture', 'Inference Inputs', 'test2017']
4.2.2 Results For our lattice transformer, we have three models trained for comparison. First we use the 32K BPE Chinese corpus alone to train our lattice transformer, which is equivalent to the standard transformer with relative positional embeddings. Secondly, we train another lattice transformer with the word lattice corpus from scratch. In addition, we follow the convention of the speech translation task in previous experiments by fine-tuning the first model with word lattice corpus. For each setting, the model evaluated on test 2017 dataset is selected from the best model performed on the dev2017 data. The fine-tuning of Lattice Model 3 starts from the best checkpoint of Lattice Model 1. Notice that the first two results of transformer-base and -big are directly copied from the relevant references. From the result, we can see that our Model 1 can be comparable with the vanilla transformer-big model in a base setting, and significantly better than the transformer-base model. We also validate the argument that training from scratch can also achieve a better result than most baselines. Empirically, we find an interesting phenomena that training from scratch converges faster than other settings.