Avoiding Echo-Responses in a Retrieval-Based Conversation System
Table 2: Evaluation results based on the context-response test set 2. RN – random negatives model; BL – RN model with responses filter; HNr – hard negatives model based on responses only; HNr+c – proposed hard negatives model based on both responses and contexts. The metrics are described in Section 3.4
['[EMPTY]', '[ITALIC] RN', '[ITALIC] BL', '[ITALIC] HNr', '[ITALIC] HNr+c']
As we can see, the proposed HNr+c model achieves the highest values in almost all metrics compared to other approaches. According to rankcontext, it turns out that this model does not tend to highly rank input contexts and have them in the top response candidates. Still, according to the diffresponse metric, the average score of a ground-truth response is lower than the score of a context, which means that the context can be ranked higher than the ground-truth response.