Robust Spoken Language Understanding via Paraphrasing
Table 4: Comparison of 10 fold average test intent accuracy and slot tagging F1 score percentages of the sequence-to-sequence model on ATIS corpus with increasing size of training set, using RASA (RASA, 2018) as the base parser.
['[BOLD] Parsers', '[BOLD] Metric', '[BOLD] Training dataset size  [BOLD] 500', '[BOLD] Training dataset size  [BOLD] 1500', '[BOLD] Training dataset size  [BOLD] 2500', '[BOLD] Training dataset size  [BOLD] 3500']
Next, we compare the performance of the sequence-to-sequence paraphrase model, when used with RASA as the base parser. RASA uses a kernel SVM classifier along with feature selection. Hence, in general it has a worse performance than attention BiRNN parser. However, it has the advantage of fast training time. Once again, we observe that the paraphrase model is able to achieve high gains in intent accuracy over the baseline RASA parser. The slot tagging performance do not improve with this model as previously observed with attention BiRNN parser. As mentioned before, due to the lack of slot tagging confidence scores in RASA, we are unable to use our RNN based paraphrase model with RASA.