Multi-Domain Spoken Language Understanding Using Domain- and Task-Aware Parameterization
Table 2: Main Results (Overall Exact, Slot and Intent denote the corresponding metrics on whole datasets and domain exact represents the exact accuracy on each domain separately). The numbers with * indicate that the improvement of our framework over all baselines is statistically significant with p<0.01 under t-test.
['Model', 'MTOD Overall Exact', 'MTOD Slot', 'MTOD Intent', 'MTOD Reminder Exact', 'MTOD Alarm Exact', 'MTOD Weather Exact', 'ASMixed Overall Exact', 'ASMixed Slot', 'ASMixed Intent', 'ASMixed ATIS Exact', 'ASMixed SNIPS Exact']
Following prior works Goo et al. ; Qin et al. We can observe that: 1) Locale-agnostic-Universal has achieved the best performances on intent detection among all baselines, which indicates that explicitly model domain-shared and domain-specific is more effective than implicitly capturing shared knowledge with sharing parameters. 2) Our model significantly outperforms all the baselines by a large margin and achieves the state-of-the-art performance. In the MTOD dataset, compared with the best prior joint work One-Net, we achieve 2.35% improvement on Reminder domain, 2.22% improvement on Alarm domain and 1.46% improvement on Weather domain. In the ASMixed dataset, the same trend has been witnessed. This indicates the effectiveness of our domain-aware and task-aware framework which can effectively improve performance for each domain. 3) To see the role of our two-stage decoder intuitively, we also present the result when using the oracle Filter information. We can see that given the oracle Filter label where we correctly mask the token that does not need domain-specific feature will lead to better slot filling performance, which further verifies the effectiveness of our two-stage decoder.