ConCET: Entity-Aware Topic Classification for Open-Domain Conversational Agents
Table 5. Topic classification on Alexa and Self-Dialogue datasets, where (S) stands for Spotlight entity linker and (P) stands for the domain-specific PMI-EL entity linker. The relative improvements over ADAN and VDCNN are shown on the Alexa and Self-Dialogue datasets, respectively.
['[BOLD] Dataset  [BOLD] Method', '[BOLD] Dataset Alexa', '[BOLD] Dataset Alexa', '[BOLD] Dataset Self-Dialogue', '[BOLD] Dataset Self-Dialogue']
The results show that both variations of ConCET outperform the state-of-the-art classifier baselines Fastext, VDCNN, and ADAN on Alexa dataset by large margins of 13%, 23%, and 10%, respectively in terms of Micro-Averaged F1 score. Among the baselines, ADAN has the best results on the Alexa dataset, while VDCNN achieves the best results on the Self-Dialogue dataset. All the improvements are statistically significant using one-tailed Student’s t-test with p-value ¡ 0.05.