Open Set Authorship Attribution toward Demystifying Victorian Periodicals
Table 3: Varying the document length in test set while it is fixed in training. Number of sentences in training documents are fixed at 50 but different lengths are considered for test documents. Numbers of sentences considered are 10, 25, 50, 100 and the entire book. Reported results are ¯F1 scores.
['| [ITALIC] D| [BOLD]  / | [ITALIC] V|', '[BOLD] 100', '[BOLD] 500', '[BOLD] 1K', '[BOLD] 5K', '[BOLD] 10K']
To simulate a scenario, where documents during test time may emerge with arbitrary lengths, we fixed document length to 50 sentences in the training phase and varied the number of sentences in test samples. The very first problem in this setting is the scaling problem between test and training documents due to different document lengths. To address this problem we normalized word counts in each document by dividing each count by the maximum count in that document. This scaling operation is followed by maximum absolute value scaling (MaxAbsScaler) on columns. MaxAbsScaler is preferred over min-max scaling to preserve sparsity as the former does not shift/center the data. Indeed, fixing document length in the training set to a number large enough to ensure documents are sufficiently informative for the classification task at hand while small enough to provide adequate number of training samples for each class yields results comparable to those achieved by varying sizes of training document length. It is interesting to note that the larger number of training samples available in this case led to improvements in the book-level attribution accuracy as all 131 test books are correctly attributed to their true authors for vocabulary size 500 or larger.