A Scalable Neural Shortlisting-Reranking Approach for Large-Scale Domain Classification in Natural Language Understanding
Table 3: The k-best classification accuracies (%) of Shortlister using different softmax functions in the traditional and large-scale IPDA settings.
['[EMPTY]', 'Traditional IPDA  [ITALIC] smxa', 'Traditional IPDA  [ITALIC] smxb', 'Large-Scale IPDA  [ITALIC] smxa', 'Large-Scale IPDA  [ITALIC] smxb']
With only 20 domains in the traditional IPDA setting, the accuracy is over 95% even when we take 1-best or top domain returned from Shortlister. The accuracy approaches 99% when we consider Shortlister correct if the ground-truth domain is present in the top 5 domains. The results suggest that the character and word-level information by itself, coupled with BiLSTMs, can already show significant discriminative power for our task.