Deriving Machine Attention from Human Rationales
Table 3: Accuracy of transferring between aspects. Models with † use labeled data from source aspects. Models with ‡ use human rationales on the target aspect.
['Source', 'Target', 'Svm', 'Ra-Svm‡', 'Ra-Cnn‡', 'Trans†', 'Ra-Trans‡†', 'Ours‡†', 'Oracle†']
Table 3 summarizes the results of aspect transfer on the beer review dataset. Our model (OURS) obtains substantial gains in accuracy over the baselines across all three target aspects. It closely matches the performance of ORACLE with only 0.40% absolute difference. [CONTINUE] Specifically, all rationale-augmented methods (RA-SVM, RA-TRANS and OURS) outperform their rationale-free counterparts on average. This confirms the value of human rationales in the low-resource settings. We observe that the transfer baseline that directly uses rationale as augmented supervision (RA-TRANS) underperforms ORACLE by a large margin. This validates our hypothesis that human rationales and attention are different.