Probing for Semantic Classes:Diagnosing the Meaning Content of Word Embeddings
Table 4: Accuracy for predicting ambiguity
['model', 'LR', 'KNN', 'MLP']
Accuracy of standard word embeddings is clearly above the baselines, e.g., 81.2% for MLP and 77.9% for LR compared to 64.8% for FREQUENCY. The figure shows that the decision becomes easier with increased ambiguity (e.g., ≈100% for 6 or more S-classes). It makes sense that a highly ambiguous word is more easily identifiable than a two-way ambiguous word. MLP accuracy for unifΣ is close to 100%. We can again attribute this to the fact that rare senses are better represented in unifΣ than in regular word embeddings, so the ambiguity classification is easier.