Neural Representation Learning for Clarificationin Conversational Search
Table 5. Results for the next clarifying question selection task, up to 3 conversation turns. † and ‡ indicate statistically significant improvements compared to all the baselines with 95% and 99% confidence intervals, respectively. ∗ indicates statistical significant improvements obtained by MTL compared to the STL training of the same model at 99% confidence interval.
['[BOLD] Method', '[BOLD] MRR', '[BOLD] nDCG@1', '[BOLD] nDCG@5', '[BOLD] nDCG@20']
The hyper-parameters of these baselines were set using cross validation, similar to the proposed method. For the proposed method we only use the top retrieved documents as the external information source, since this is the most realistic setting for selecting clarifying questions. The proposed model significantly outperforms all baselines, including the recent BERT-NeuQS model. Consistent with the document retrieval experiments, the multi-task learning setting led to higher performance compared to STL. In addition, the achieved performance compared to the original query shows the benefit of asking clarification.