Dynamic Natural Language Processing with Recurrence Quantification Analysis
Table 2: Means of RQA measures for the various genres extracted from the Gutenberg dataset (gutenbergr).
['Genre', 'N', '[ITALIC] RR', '[ITALIC] DET', '[ITALIC] MAXLINE', '[ITALIC] MEANLINE', '[ITALIC] ENT']
Using the library gutenbergr, we extracted 6,408 texts from the most frequent 20 subjects. We then extracted a segment of text of 5,000 words, from the 5,000th to the 10,000th word. This was to ensure that we did not conduct RQA over the table of contents of the Gutenberg works. This facilitates analysis under limitations of memory, but it also ensures that genres are compared on approximately the same length of text. Much like the example in the prior section, all texts were subjected to RQA. There is clear clustering when we look at the way that genres behave under different measures. An example is shown in Fig. Poetry and fairy tales are in a separate cluster relative to the majority of the genres, with the general class of “prose fiction” seeming to reflect the mass of this distribution. However subtle features of the dendrogram suggest genres cluster together in intuitive ways. For example, “juvenile fiction” clusters in one leaf, and adventures and westerns are close in the tree. It is important to note that some of these results may be due to works falling under multiple categories. However, this only represents a small percentage of the overall distribution. Out of the approximately 6,400 texts, only 16% are listed under two categories, and only 2% under three. All measures are significantly accounted for by genre category. For each measures, RR, DET, etc., we build a separate regression model that predicts the value of those measures from 20 genres in the form of dichotomous variables. All measures significantly vary in some manner around genre category, with the most variance accounted for in the RR,DET,MEANLINE and ENT measures, all over 10% of variance accounted for by the 20 genre codes.