Alleviating Overfitting for Polysemous Words for Word Representation Estimation Using Lexicons
TABLE III: Correct answer rates of word analogical reasoning tasks when paraphrases of different types were used. “Entailment” set in every test includes “forward entailment” and “reverse entailment.” “Sem” refers to the semantic part. “Syn” refers to the syntactic part.
['Paraphrase Set', 'Sem [%]', 'Syn [%]', 'Total [%]']
We used paraphrases of different types to train 100-dimensional word vectors in the hidden layer with text8. The context window was eight. The threshold θ is set to 3.5. Then we used the trained word vectors to do the word analogical reasoning task. Employing paraphrases of “other” type together with “equivalence” and “entailment” achieves better performance in syntactic tests, best total correct answer rates but is weaker in semantic tests. Adding paraphrases of “exclusive” type improves the correct answer rate of semantic tests but deteriorate that of syntactic tests. Involving paraphrases of “independent” type improves the correct answer rate of syntactic tests but deteriorate that of semantic tests.