Building a Production Model for Retrieval-Based Chatbots
Table 4: Recall@k from n response candidates for different values of n using random whitelists. Each random whitelist includes the correct response along with n−1 randomly selected responses.
['[BOLD] Candidates', '[BOLD] R@1', '[BOLD] R@3', '[BOLD] R@5', '[BOLD] R@10']
Table 4 shows Rn@k on the test set for different values of n and k when using a random [CONTINUE] Table 4 shows that recall drops significantly as n grows, meaning that the R10@k evaluation performed by prior work may significantly overstate model performance in a production setting.