Vector Embedding of Wikipedia Concepts and Entities
Table 3: Comparing the results in Phrase Similarity dataset. Rho is Spearman’s correlation to the human evaluators. !Found is the number of pairs not found in each approach dataset.
['Datasets #', 'Datasets Dataset Name', 'Datasets #Pairs', 'Wikipedia Miner !Found', 'Wikipedia Miner Rho', 'Google Freebase !Found', 'Google Freebase Rho', 'ConVec !Found', 'ConVec Rho', 'ConVec (Heuristic) !Found', 'ConVec (Heuristic) Rho']
In these datasets, each row consists of two words with their relatedness assigned by the human. The Spearman’s correlation is used for comparing the result of different approaches with the human evaluated results. These datasets contain words and not the Wikipedia concepts. We replaced all the words in these datasets with their corresponding Wikipedia pages if their surface form and the Wikipedia concept match. We used the simple but effective most frequent sense disambiguation method to disambiguate words that may correspond to several Wikipedia concept. This method of assigning words to concepts is not error prone but this error is considered for all approaches. The average correlation for the heuristic based approach is less than the other approaches, but average of not-found entries in this approach is much less than the others. It shows that using the heuristic can increase the coverage of the Wikipedia concepts.