Meta Answering for Machine Reading
Table 5: Overall accuracy for the original model and the agent, split by action type.
['[EMPTY]', 'ABSTAIN Correct', 'ABSTAIN Incorrect', 'ANSWER Correct', 'ANSWER Incorrect']
We find that MMA\textscAnswerOnly is a slightly more accurate abstainer, but that by and large, its tendency to answer or abstain matches that of BERTWWM. However, it is a considerably more accurate answerer – even though it answers 38 times fewer than BERTWWM, it increases the absolute number of correct answers given to by 83 and decreases the number of incorrect answers given by 121, explaining both the higher precision and recall for the NQ evaluation metric.