“None of the Above”:Measure Uncertainty in Dialog Response Retrieval
Table 2: Results for 2,5,10,20,40,60,80,100 candidate responses with the original selection model
['50% NOTA Test Results On More Distractors (%) #Candidates', '50% NOTA Test Results On More Distractors (%) R', '50% NOTA Test Results On More Distractors (%) N', '50% NOTA Test Results On More Distractors (%) N F1', '50% NOTA Test Results On More Distractors (%) G F1', '50% NOTA Test Results On More Distractors (%) Average F1']
The direct predict model is run as it does not need further tuning. Threshold approach, especially with softmax probability as threshold, will need separate rounds of tuning on the threshold. Here, the average F1 is averaged on all 4 F1 scores. For each model architecture, the best performing setting for each metric is in bold.