An Efficient Approach to Encoding Context for Spoken Language Understanding
Table 1: SLU results on test sets with baselines and our proposed architecture variants, when trained on Sim-M + Sim-R. For each dataset, the columns indicate the intent accuracy, dialogue act F1 score, slot chunk F1 score and frame accuracy, in that order. The Config column indicates the best obtained config for feeding context vectors for each experiment.
['[BOLD] Model', '[BOLD] Config', '[BOLD] Config', '[BOLD] Sim-R Results Intent', '[BOLD] Sim-R Results Act', '[BOLD] Sim-R Results Slot', '[BOLD] Sim-R Results Frame', '[BOLD] Sim-M Results Intent', '[BOLD] Sim-M Results Act', '[BOLD] Sim-M Results Slot', '[BOLD] Sim-M Results Frame', '[BOLD] Overall Results Intent', '[BOLD] Overall Results Act', '[BOLD] Overall Results Slot', '[BOLD] Overall Results Frame']
[leftmargin=10pt, topsep=0pt, noitemsep] at only, No DE : We feed at, the system act encoding, in one of positions A-D, omit the dialogue encoder, and instead use ut, the utterance encoder’s final state, for intent and act prediction. ot−1 shows results for the best model with ot−1 fed in position D. at and ot−1 : We feed at into the dialogue encoder, and ot−1 and at independently into one of positions C and D, 4 combinations in total. The utterance encoder and slot tagger layer sizes are set equal to the token embedding dimension, and that of the dialogue encoder to half this dimension. A frame is considered correct if its predicted intent, slots and acts are all correct. We observe that our models compare favorably to the state of the art MemNet and SDEN baselines. The use of context plays a crucial role across all datasets and tasks, especially for intent and dialogue act classification, giving an improvement of ∼15% and ∼5% respectively across all configurations. For all subsequent discussion, we concentrate on frame accuracy since it summarizes the performance across all tasks. In general, we observe that feeding context vectors as initial states to bidirectional RNNs (i.e. position D for slot tagging plus a side input to the dialogue encoder, or position B for all tasks in case of no dialogue encoder) yields better results than feeding them as additional inputs at each RNN step (positions C and A). This may be caused by the fact that our context vectors do not vary with the user utterance tokens, because of which introducing them repeatedly is likely redundant.