When Choosing Plausible Alternatives, Clever Hans can be Clever
Table 3: Results of human performance evaluation of the original COPA and Balanced COPA.
['Dataset', 'Accuracy', 'Fleiss’ kappa  [ITALIC] k']
The human evaluation shows that our mirrored instances are comparable in difficulty to the original ones (see Table 3).