Open Set Authorship Attribution toward Demystifying Victorian Periodicals
Table 2: ¯F1 scores from closed set experiment with 36 authors. Both training and test documents have the same length in terms of the number of sentences. Numbers of sentences considered are 10, 25, 50, 100 and entire book. Columns represent different vocabulary sizes. 100, 500, 1000, 5000 and 10000 most frequent words are considered.
['| [ITALIC] D| [BOLD]  / | [ITALIC] V|', '[BOLD] 100', '[BOLD] 500', '[BOLD] 1K', '[BOLD] 5K', '[BOLD] 10K']
Not surprisingly the AA prediction performance improves as the document length increases. Rate of improvement is more significant for shorter document lengths. Using entire novel as a document slightly hurts the performance as the number of training samples per class dramatically decreases. The same situation is also observed with the size of the feature vector. The rate of improvement is more remarkable for smaller vocabulary sizes. These results suggest that a vocabulary size of around 1000 and a document length of around 50 sentences can be sufficient to distinguish among eminent novelists with near perfect accuracy if three or more of their books are available for training.