ASSESSING THE TOLERANCE OF NEURAL MACHINE TRANSLATION SYSTEMS AGAINST SPEECH RECOGNITION ERRORS
Table 4: Changes in sentence-level TER rankings as ASR errors are introduced.
['Gold Winner', 'ASR Winner', 'Absolute %', 'Relative %']
Across all ASR outputs, 70.2% of the MT evaluation ranking decisions remain the same when ASR errors create noisy input. The neural model retains a higher rank 7.5% more often than mmt as ASR errors are introduced. Ranking ties remain 55.5% of the time. Of the remaining, the neural model outperforms mmt 5.5% more often in the presence of ASR errors (25.0% versus 19.5%). These results confirm that at the corpus level NMT produces higher scoring translations in the presence of ASR errors.