Adapting predominant and novel sense discovery algorithms for identifying corpus-specific sense differences
Table 3: Number of candidate words using Lau’s method.
['[EMPTY]', '1987-1995', '2006-2008']
Since Lau’s method is computationally expensive to run over the whole vocabulary, we run it only for those candidate words, which were flagged by Mitra’s method. We then use a threshold to select only those topics which have low similarity to all the topics in the other corpus. We use 0.35 as the threshold for all the 4 cases except for news-specific senses in 2006-2008, where a threshold of 0.2 was used. Note that a word may have multiple corpus-specific senses.