Building a Production Model for Retrieval-Based Chatbots
Table 3: AUC and AUC@p of our model on the propriety help desk dataset.
['[BOLD] Metric', '[BOLD] Validation', '[BOLD] Test']
The performance of our model according to these AUC metrics can be seen in Table 3. The high AUC indicates that our model can easily distinguish between the true response and negative responses. Furthermore, the AUC@p numbers show that the model has a relatively high true positive rate even under the difficult requirement of a low false positive rate.