Jointly Trained Transformers models for Spoken Language Translation
Table 3: Cascaded Sp-MT models trained using Transformer.
['Architecture (Input-Output) Granularity', '1-best Dev set', '1-best Test set', 'n-best Dev set', 'n-best Test set', 'ranked n-best Dev set', 'ranked n-best Test set']
Cascaded SLT comprises of an ASR system followed by an MT. Two different cascades systems have been trained i.e., ASR systems with characters, BPE’s as output units along with the corresponding MT model. The performance of cascaded SLT system is presented in the Table. The ASR hypothesis is decoded and the decoded hypothesis is used by the MT model to produce the translation. Due to the use of ASR hypothesis as input text the performance of MT systems a degradation in the BLEU ¿ 10 has been observed. Column 1 is the cascade-SLT system, Column 2, 3 is the performance of SLT systems using n-best hypothesis. Columns 4, 5 are the performance of SLT systems which uses the n-best hypothesis which are ranked based on the log-likelihood scores. The per-sentence log-probabilities from the n-best ASR hypothesis is used as a ranking score. The prefixes that correspond to the ASR hypothesis are initialized with their corresponding n-best ranking score. The general pipeline search is described below: P(y|x)=argmaxy∑z∈^Z(x))P(y|z)P(z|x)))≡argmaxy∑z∈^Z(x))log(P(y|z))+log(P(z|x)))) (7) In the above equation, x is source speech and y is the target token sequence and z is the source token sequence. Using the N-best hypothesis without ranking scores from the ASR has degraded the performance which can be seen in column 3,4. Using N-best hypothesis with ranking scores has improved the performance.