Bootstrapping Multilingual Intent Modelsvia Machine Translation for Dialog Automation
Table 3: Machine translation quality measured in BLEU, TER, and utterance length, evaluated against post-edited translations of human transcripts.
['Translation', 'BLEU ↑', 'TER ↓', 'Length']
ASR errors increase the required translation edits by 10%, from 41.3% edits to 51.2%. The primary sources of errors are incomplete sentences, lack of punctuation; lexical mistranslations of key words: (e.g. equipo (equipment)→team; dirección (address)→direction; reclamo (complaint)→ claim); ambiguous translations (e.g. factura→bill/invoice); and duplicated words during translation (e.g. payment arrangements→payment payment). Many of these issues are due to lack of in-domain MT training data and low tolerance of ASR errors. Highly repetitive errors indicate that an automatic post-editing system could substantially improve the translation system’s quality.