End-to-End Slot Alignment and Recognition for Cross-Lingual NLU
Table 3: Zero-shot results on MultiATIS++ averaged over 5 runs. The No MT rows are models trained only on the English data. The MT + project rows correspond to models trained on the English and machine translated data with automatically projected slot labels using fast-align. The MT + soft-align row is the model trained on the English and machine translated data using our soft-alignment method.
['[BOLD] Intent acc. No MT', '[BOLD] Intent acc. LSTM', 'es 64.82', 'de 64.77', 'zh 59.69', 'ja 65.40', 'pt 65.15', 'fr 69.92', 'hi 60.11', 'tr 63.64']
but Turkish – a dissimilar language to English. For both LSTM and BERT models, adding MT data with projected slot labels brings significant improvements on intent accuracy – by 13–33% for LSTM and 1–24% for BERT. However, we observe different trends on slot F1 for different languages. For example, when using BERT, adding MT data improves slot F1 by 20–44% on Chinese, Japanese, and Hindi, but hurts by around 6% on French. This may be due to its mixed effect – training directly on the target language data is beneficial especially for languages dissimilar to English, however, the noisy projection of the slot labels could also bring harm to the model.