SemRe-Rank: Improving Automatic Term Extraction By Incorporating Semantic Relatedness With Personalised PageRank
Table 6. Number of rare RTPs (Recoverable True Positives) compared to the total number of RTPs found in the candidate term lists of each ATE method. A rare RTP is defined as one whose composing words all have a total corpus frequency of less than 5.
['Dataset', 'Basic, ComboBasic, LP, NTM, PU Rare RTP', 'Basic, ComboBasic, LP, NTM, PU Total RTP', 'TFIDF, CValue, RAKE, Weirdness, Relevance, GlossEx,  [ITALIC] χ2 Rare RTP', 'TFIDF, CValue, RAKE, Weirdness, Relevance, GlossEx,  [ITALIC] χ2 Total RTP']
First, we aim to understand for a given dataset, the extent to which rare words are used as part (or whole) of real terms. For this we quantify the number of ‘rare’ RTP’s found in the candidate terms extracted by each ATE method for each dataset. A rare RTP is one whose composing words are all ‘rare words’. We call a word ‘rare’ if it has a total corpus frequency below 5, which is the default parameter used in the word2vec implementation to discard any infrequent words. We consider this a minimum requirement for learning ‘reasonably quality’ word embedding vectors. Although they represent only a small percentage, this confirms that rare words can potentially impact on SemRe-Rank because they can be used in real terms.