Improving Correlation with Human Judgmentsby Integrating Semantic Similarity with Second–Order Vectors
Table 3: Threshold Correlation with vector-faith
['T', '# bigrams', 'UMNSRS sim', 'UMNSRS rel', 'MiniMayoSRS MD', 'MiniMayoSRS coder']
Although, unlike res whose scores are greater than or equal to 0 without an upper limit, the faith measure returns scores between 0 and 1 (inclusive). Therefore, here a threshold of 0 indicates that all of the bigrams were included in the similarity matrix; and a threshold of 0.1 indicates that only the bigram pairs with a similarity score greater than 0.1 were included. The results show an increase in accuracy for all of the datasets except for the MiniMayoSRS tagged for physicians. The increase in the results for the UMNSRS tagged for similarity and the MayoSRS is statistically significant (p≤0.05). This is not the case for the UMNSRS tagged for relatedness nor the MiniMayoSRS.