Bootstrapping Multilingual Intent Modelsvia Machine Translation for Dialog Automation
Table 4: Intent classification performance by machine learning models and human intent analysts (IAs) at 0%, 10%, and 20% rejection.
['Configuration', 'Reject 0%', 'Reject 10%', 'Reject 20%']
We compare the results of each bootstrapped NLU approach with a native Spanish intent model trained on the held-out Spanish training data. We additionally repeat the experiment with the human transcripts to measure the difference in intent classification error that may be explained by ASR. Error-rejection curves for each intent model are shown in Fig. Our reference native Spanish intent classification model is trained on ASR outputs since an insufficient amount of human-transcribed intent modeling data is available. At the same time, if the IAs are presented with the ASR outputs, they produce an error rate of 24.4%. This demonstrates the intent model’s ability to tolerate a certain degree of ASR errors by being trained on ASR errors. Finally, we measure IA labeling performance on translated utterances. In our conventional scenario, intent analysts listen to audio segments in their native language and provide an intent label. Instead, we replace the original audio with machine translated or translation post-edits of ASR hypotheses. Fig.