Guessing What’s Plausible But Remembering What’s True: Accurate Neural Reasoning for Question-Answering
Table 1: Precision, recall and F1 for model jointly trained on all four reasoning tasks.
['Set', 'R', 'k=1 34.7', 'k=10 94.7', 'k=100 98.9', 'k=1000 99.8']
We measured the precision, recall and F1 of the top k values in each computed set (averaged across the test cases). The sketches ensure that sets are high precision (for this sketch size, precision is always 99% or better), but the experiments verify that training can make the sets coherent, so relatively small k can be used while still obtaining good recall.