Building Chatbots from Forum Data:Model Selection Using Question Answering Metrics
Table 2: MAP score for the ranking strategies defined in Section 4.2, evaluated on the development and on the test datasets.
['[BOLD] Ranking Metric', '[BOLD] MAP  [BOLD] Dev', '[BOLD] MAP  [BOLD] Test']
The results are consistent on the development (63.56) and on the test datasets (63.45). The absolute improvement with respect to cos-embeddings+qc-sim is +0.59 on the development and +0.55 on the test dataset, respectively.