Authorship Attribution Using the Chaos Game Representation
Table 3: Average accuracy results for the classification on validation and test sets over 30 trials, each with random samples in the training and validation sets. The test set comprises the disputed essays and the ground truth is considered with Madison as author.
['Classifier', 'Validation set 1 (%)', 'Validation set 2 (%)', 'Test set (%)']
Accordingly, the parameter settings of chunk size 8500 and k=7 are used for the subsequent test on the Federalist Papers, while randomly varying the training and validation samples in 30 different repeated runs. The aim is to ascertain that the methods are not sensitive to the choice of used training and validation sets. It can be observed that all of them have higher classification accuracy on the first validation set as compared to the second one: the former is obtained by gathering the FCGR images produced from the final chunk of each essay from the training set, therefore it is natural that those text parts resemble more the rest of the essays that remained in the training sets. As concerns the test set, in all trials, 10 of the 12 essays are attributed to Madison by both classifiers. The Federalist Papers 62 and 63 are the ones that, in some trials, get attributed to Hamilton. For LR this happens in 2 of the 30 trials for essay 62, and in 1 trial for essay 63. For SVM the situation is quite different: in 24 times of the 30 trials the essay 62 is attributed to Hamilton, and in 25 trials the same situation occurs for essay 63. The results of FTT+PCA are more similar to those of the SVM: in 27 of 30 trials it assigns essay 62 to Hamilton and in 29 cases essay 63 is labeled also with Hamilton. As concerns the other disputed essays, there is only one minor exception for essay 50, which is assigned by FTT+PCA in 1 trial out of 30 to Hamilton. historians feel weakest about”, so it can be observed that the results of the proposed methodology get complicated for the exact essays that are considered tricky.