MCScript2.0: A Machine Comprehension Corpus Focused on Script Events and Participants
Table 2: Accuracy on test set, and on script/text-based questions (accscr, acctxt) on MCScript2.0. The maximum per column is printed in bold.
['[EMPTY]', '[BOLD] \xa0acc', '[BOLD] acc [ITALIC] scr', '[BOLD] acc [ITALIC] txt']
Overall Performance. As can be seen, the logistic model scores worst, presumably because it has been used for the adversarial filtering algorithm and the data are thus most challenging for this model. TriAN performs best, clearly outperforming the attentive reader. TriAN is apparently superior in its way of text processing, since it employs a richer text representation and exploits attention mechanisms on more levels, which is reflected by a higher accuracy on text-based questions. In contrast, script-based questions seem to be challenging for TriAN. This is interesting, because it shows that ConceptNet alone cannot provide sufficient information for answering the kind of questions that can be found in MCScript2.0.