On the Feasibility of Automated Detection of Allusive Text Reuse
Table 2: Retrieval results for all considered models grouped by approach type. All models are evaluated with tokens and lemmas as input except for SCwn which requires lemmatized input. Overall best numbers per metric are shown in bold letters.
['Metric', 'Lemma', 'Lexical BOW', 'Lexical TfIdf', 'Lexical Tesserae', 'Semantic BOWemb', 'Semantic TfIdfemb', 'Semantic WMD', 'Hybrid SCwn', 'Hybrid SCemb', 'Hybrid T+WMD']
Interestingly, a simple TfIfd baseline over lemmatized input results in strong ranking performance, surpassing all other purely lexical -- including the hand-crafted Tesserae -- and all purely semantic models. In agreement with general expectations, all models benefit from lemmatized input and TfIdf transformation (both as input representation in purely lexical models and as a weighting scheme for the sentence embeddings in purely semantic approaches). WMD outperforms any other purely semantic model, but as already pointed out, it compares negatively to the purely lexical TfIdf baseline. The combination of Tesserae with WMD as back-off proves useful and outperforms both approaches in isolation, highlighting that they model complementary aspects of text reuse.