De-Conflated Semantic Representations
Table 5: Evaluation results on the word to sense similarity test dataset of the SemEval-14 task on Cross-Level Semantic Similarity, according to Pearson (r×100) and Spearman (ρ×100) correlations. We show results for four similarity computation strategies (see §3.3). The best results per strategy are shown in bold whereas they are underlined for the best strategies per system. Systems marked with ∗ are evaluated on a slightly smaller dataset (474 of the original 500 pairs) so as to have a fair comparison with RotheSchutze:2015 and Chenetal:2014 that use older versions of WordNet (1.7.1 and 1.7, respectively).
['[BOLD] System', '[BOLD] MaxSim  [ITALIC] r', '[BOLD] MaxSim  [ITALIC] ρ', '[BOLD] AvgSim  [ITALIC] r', '[BOLD] AvgSim  [ITALIC] ρ', '[BOLD] S2W  [ITALIC] r', '[BOLD] S2W  [ITALIC] ρ', '[BOLD] S2A  [ITALIC] r', '[BOLD] S2A  [ITALIC] ρ']
As can be seen from the low overall performances, the task is a very challenging benchmark with many WordNet out-of-vocabulary or slang terms and rare usages. Despite this, DeConf provides consistent improvement over the comparison sense representation techniques according to both measures and for all the strategies.