Domain-based Latent Personal Analysis and its use for impersonation detection in social media
Table 4: Validation and sensitivity: Books dataset
['Experiment', 'Description', 'Average', 'Median', 'Std.']
The above selection process yielded KLDε as the chosen metric. We would like to verify that a sample text by an author is significantly closer to other texts of that author than to texts written by others. We use the Books dataset to validate our method. This set of labeled data allows us to experiment with our method and draw conclusions regarding its strengths and weaknesses with labeled domain data. Each book is thus a domain, and the chapters are snippets drawn from the domain. To validate that chapters are closer to the book they were taken from, we also create virtual books constructed by randomly chosen chapters from a variety of books written by different authors. We construct each book as a DVR and its sample chapters as PVR’s. We hypothesize that the mean distance between the chapters’ PVR and the authentic book DVR is significantly smaller than that of chapters’ PVR and virtual books’ DVR. This will indicate that a sample of an author’s writing is significantly closer to a larger document written by them than to a document written by several people, i.e. that their choice of words is consistent. We perform a sequence of three experiments, controlling first for the book and chapter lengths, and with each experiment, we relax length restrictions, to see the validity of the method for texts of different sizes, adding a sensitivity test. The results are averaged over 30 real books and 30 virtual books. Each real book contains between 7 to 61 chapters. Virtual books were created with a randomly chosen number of chapters from the pool of real books. Each virtual book contains 19 to 31 chapters. Our null hypothesis is that there will be no statistically significant difference between the mean distance of a chapter to a real book written by the author, and the mean distance of a chapter to a virtual book.