Robust Spoken Language Understanding via Paraphrasing
Table 5: Comparison of 10 fold average test intent accuracy percentage of all our models on simulated ATIS log corpus with increasing size of training set, using both attention BiRNN (Liu and Lane, 2016) and RASA (RASA, 2018) as the base parser. The models are trained on original ATIS dataset but tested on ATIS log corpus.
['[BOLD] Parsers', '[BOLD] Training dataset size  [BOLD] 500', '[BOLD] Training dataset size  [BOLD] 1500', '[BOLD] Training dataset size  [BOLD] 2500', '[BOLD] Training dataset size  [BOLD] 3500']
Finally, to validate the robustness of our models in an adversarial post deployment scenario, we test the performance of our models on the simulated ATIS log corpus. Due to a distribution mismatch with the ATIS training data, all models perform worse in this dataset, as expected. However, we still observe that, irrespective of the base parser used, our paraphrase models achieve an improved intent detection accuracy.