Dual Learning for Semi-Supervised Natural Language Understanding
TABLE VII: Ablation studies of the dual pseudo-labeling method.
['[BOLD] Method', '[BOLD] SNIPS (5%) Slot F1', '[BOLD] SNIPS (5%) Intent Acc']
The dual pseudo-labeling method creates pseudo-samples in two ways: a) obtaining predicted semantic forms of sentences with the NLU model and b) generating sentences with the SSG model given intents and slot-value pairs. From the results, we can find that pseudo-samples created from SSG are more essential. The reason may be that the SSG model tends to generate sentences semantically consistent with the given semantic forms though the sentences are not natural enough. However, the NLU model may predict wrong semantic labels. Meanwhile, without pseudo-samples from SSG model, the dual pseudo-labeling method reduces to the traditional pseudo-labeling without dual task. We believe that the NLU and SSG models updated with more iterations could provide more qualified pseudo-samples. Therefore, the performance decreases if we keep using pseudo-samples generated at the first iteration, as shown in “(-) w/o iterative generation” row.