XL-NBT: A Cross-lingual Neural Belief Tracking Framework
Table 2: Experimental results for cross-lingual NBT and other baseline algorithms. All results are averaged over 5 runs. Here we use “mono” to refer to the experiments with pre-trained monolingual embedding, “bilingual” to refer to the experiments with pre-trained bilingual embedding.
['[BOLD] Language Models', '[BOLD] Language Models', '[BOLD] German (student) Goal', '[BOLD] German (student) Request', '[BOLD] Italian (student) Goal', '[BOLD] Italian (student) Request', '[BOLD] English (teacher) Goal', '[BOLD] English (teacher) Request']
Here we highlight the baselines we use to compare with our cross-lingual algorithm as follows: (1) Supervised: this baseline algorithm assumes the existence of annotated dialog belief tracking datasets, and it determines the upper bound of the DST model. (2) w/o Transfer: this algorithm trains an English NBT, and then directly feeds target language into the embedding level as inputs during test time to evaluate the performance. (3) Ontology-match: this algorithm directly uses exact string matching against the utterance to discover the perceived slot-value pairs, it directly assigns a high score to the appearing candidates. (4) Translation-based : this system pre-trains a translator on the external bilingual corpus and then translates the English dialog and ontology into target language as “annotated” data, which is used to train the NBT in the target language domain (more details about the implementation, performance and examples are listed in the appendix). (5) Word-By-Word (WBW) : this system transforms the English dialog corpus into target language word by word using the bilingual dictionary, which is used to train the NBT in target side. our cross-lingual NBT framework is able to achieve significantly better accuracy for both languages in both parallel-resource scenarios.