A Step Towards InterpretableAuthorship Verification
Table 6: Evaluation results of TAVeer and the ten selected baseline methods. Bold and underlined values represent the best and second best results.
['[EMPTY]', '[BOLD] Method', '[BOLD] Acc.', '[BOLD] AUC', '[BOLD] TP', '[BOLD] FN', '[BOLD] FP', '[BOLD] TN']
After we have trained TAVeer and the ten selected baselines on the four training corpora, we evaluated all AV methods on the respective test corpora. As can be seen in this table, TAVeer outperforms all ten baselines on the demanding corpus \mathcal{C}_{\mathrm{Stack}}, which demonstrates its robustness under cross-topic conditions. In addition, TAVeer surpasses all baseline methods on \mathcal{C}_{\mathrm{Amazon}} and is ranked second on \mathcal{C}_{\mathrm{Reddit}}, indicating that the method is also suitable for mixed-topic corpora. On the corpus \mathcal{C}_{\mathrm{Yelp}}, TAVeer is on par with the strongest baseline IM, which shows that the method is also applicable in scenarios where the documents consist of a few sentences. It should be emphasized, that half of the baseline methods (\textsf{IM}\mbox{},\textsf{ProfAV}\mbox{},\textsf{OCCAV}\mbox{} and NNCD) make use of character n-grams which, according to the literature, represent the strongest features in the field of AV. In this regard, the results demonstrate that punctuation- and TA-based features alone, are similarly effective and in some cases even better. To gain an insight into what led to these results, we therefore conduct a more detailed analysis regarding our approach. For this purpose, we first take a closer look at the models TAVeer learned on the basis of the training corpora, in order to understand their behavior on the test corpora. Using the proposed interpretation technique (cf. we then perform a more fine-grained analysis in which we investigate which specific features contributed to TAVeer’s predictions.