CAsT 2019: The Conversational Assistance Track Overview
Table 4. Automatic response retrieval results.
['[BOLD] Run', '[BOLD] Group', '[BOLD] MAP', '[BOLD] MRR', '[BOLD] NDCG@3']
Automatic run results. The results show systems that vary widely in effectiveness. The median NDCG@3 score of the automatic runs is 0.286. The best performing run not utilizing BERT for reranking is clacBase with a NDCG@3 value of 0.360. We observe that nine of the top ten best performing runs use a form of BERT for ranking results. The top two performing teams perform contextual query rewriting and expansion.