Modeling Word Emotion in Historical Language: Quantity Beats Supposed Stability in Seed Word Selection
Table 4: Results of the synchronic evaluation in Pearson’s r averaged over all three VAD dimensions. The best system for each seed lexicon and those with statistically non-significant differences (p ≥ 0.05) are in bold.
['Induction Method', 'Seed Selection', 'SVDPPMI', 'SGNS']
SGNS embeddings are worse than SVDPPMI embeddings for both full and limited seed lexicons. SVDPPMI embeddings seem to be better suited for induction based on the full seed set, leading to the highest observed correlation with ParaSimNum. However, results with other induction algorithms are not significantly different. For the limited seed set, consistent with claims by Hamilton et al. However, all results with the limited seed set are far (and significantly) worse than those with the full seed lexicon. For English, using the full seed lexicons, we achieve performance figures around r=.35. In contrast, using the limited seed lexicon we find that the performance is markedly weaker in each of our six conditions compared to using the full seed lexicon. This observation directly opposes the claims from Hamilton et al.