A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding
Table 3: The SLU performance on baseline models compared with our Stack-Propagation model on two datasets.
['[BOLD] Model', '[BOLD] SNIPS Slot (F1)', '[BOLD] SNIPS Intent (Acc)', '[BOLD] SNIPS Overall (Acc)', '[BOLD] ATIS Slot (F1)', '[BOLD] ATIS Intent (Acc)', '[BOLD] ATIS Overall (Acc)']
From the result of gate-mechanism row, we can observe that without the Stack-Propagation learning and simply using the gate-mechanism to incorporate the intent information, the slot filling (F1) performance drops significantly, which demonstrates that directly leverage the intent information with Stack-Propagation can improve the slot filling performance effectively than using the gate mechanism. Besides, we can see that the intent detection (Acc) and overall accuracy (Acc) decrease a lot. We attribute it to the fact that the bad slot filling performance harms the intent detection and the whole sentence semantic performance due to the joint learning scheme. This shows that Stack-Propagation model can learn the correlation knowledge which may promote each other and ease the error propagation effectively. 1) We conduct the sentence-level intent detection in intent detection separately, which utilizes the last hidden vector of BiLSTM encoder for intent detection. We refer it to lstm + last-hidden. From the result, we can see that the token-level intent detection obtains better performance than the utterance-level intent detection. We believe the reason is that intent prediction on each token has similar advantage to ensemble neural networks, which can reduce the predicted variance to improve the intent performance. As a result, our framework can provide more useful intent information for slot filling by introducing token-level intent detection. We can observe the self-attention mechanism can further improve the SLU performance. We attribute this to the fact that self-attention mechanism can capture the contextual information for each token. Without the self-attention mechanism, it will harm the intent detection and have bad influence on slot filling task by joint learning.