Gated Convolutional Bidirectional Attention-based Model for Off-topic Spoken Response Detection
Table 4: The comparison of different models based on over 0.999 on-topic recall on seen and unseen benchmarks. AOR means Average Off-topic Recall (%) and PRR3 means Prompt Ratio over off-topic Recall 0.3 (%).
['Systems', 'Model', 'Seen PPR3', 'Seen AOR', 'Unseen PPR3', 'Unseen AOR']
As is shown in To make the evaluation more convincing, we built a stronger baseline model G-Att-RNN based on Att-RNN by adding residual connections with each layer. Additionally, we add a gated unit as the relevance layer for our baseline model G-Att-RNN. Compared with Att-RNN, our baseline model G-Att-RNN achieved significant improvements on both seen (by +3.2 PPR3 points and +4.6 AOR points) and unseen benchmark (by +22.0 PPR3 points and +17.1 AOR). +24.0 AOR points on the unseen benchmark, as well as +9.0 PPR3 points and +7.0 AOR points on the seen benchmark. Meanwhile, our approach significantly outperforms G-Att-RNN by +14.0 PPR3 points and + 6.9 AOR points on the unseen benchmark, as well as +5.8 PPR3 points and +2.4 AOR points on the seen benchmark.