Emulating Human Conversations using Convolutional Neural Network-based IR
Table 2: Mean scores assigned by judges, together with 95% confidence intervals. The best-performing baseline and experimental systems are shown in bold.
['[BOLD] Technique', '[BOLD] Score', '[BOLD] 95% CI']
We report the results of human judgments in two ways. First, we compute scores for each of the 1000 responses on a 5 point scale: we convert the binary decisions into a score between 0 and 5 by summing the judgments, then average over those scores. Two-tailed t-tests indicate that the difference in means between the top-scoring baseline (DCGM-II+CMM) and the two SemRel(M,R) systems is statistically significant (p < 0.0001). The SemRel(M,R)+CMM+CCF appears to be slightly better overall; CCF provides a small but useful boost.