Jointly Trained Transformers models for Spoken Language Translation
Table 2: MT systems trained using Transformer Models.
['[EMPTY]', 'Dev Sacre-BLEU', 'Test Sacre-BLEU']
Transformer models have been used for training Machine Translation (MT) systems. MT systems are trained with three different input-output granularities such as characters-characters, characters-BPE and BPE-BPE. The details about the architecture of the networks are described as follows: The model has 6-encoder layers, 6-decoder layers and the dimensionality of dmodel and dff are 512, 1024 respectively. The models are trained with 8 parallel heads. The models are trained for 150 epochs and each batch comprised of 7000 target units. The model from the last ten epochs are averaged and the averaged weights are used for decoding. The beam size and length peanlities for decoding are derived from the development set. The optimal beam size for all the granularities is 5, but the length penalties has varied, for characters-characters, characters-BPE and BPE-BPE the length penalties are 1.2, 1, 1.2 respectively. The tokens predicted from the model are converted back to the text and the Sacre BLEU is computed. The performances of the MT systems trained are presented in Table. The charecter based models have taken longer time to train and decode compared to the BPE models. The models with character input and BPE as output has performed better than the other two systems.