Decay-Function-Free Time-Aware Attention to Context and Speaker Indicator for Spoken Language Understanding
Table 3: SLU accuracy of possible combinations of the proposed methods in role-level attention with different history representations. Int. used intent vector; Dist. used distance vector. ∗: p<0.05; ∗∗, p<0.01 compared to using intent only. Other codes are as in Table 2.
['[BOLD] Attention Type', '[BOLD] F1 score  [BOLD] Intent only', '[BOLD] F1 score  [BOLD] Int. & Dist.']
Although we propose to use both intent and distance by concatenating them as a history representation, intent can be used alone; this approach is more intuitive than using both intent and distance. The results indicate that distance embeddings are helpful both for attention and for the history representation. Decay-Function-Free Time-Aware Attention achieved the biggest improvement (row 5) among all the models.