Parsimonious Morpheme Segmentation with an Application to Enriching Word Embeddings
TABLE III: Word analogies.
['[BOLD] Dataset  [BOLD] Method', '[BOLD] English  [BOLD] Sem', '[BOLD] English  [BOLD] Syn', '[BOLD] German  [BOLD] Sem', '[BOLD] German  [BOLD] Syn', '[BOLD] Turkish  [BOLD] Sem+Syn']
This does not extend to semantic analogies whereby utilizing subword-information seems to cause a deterioration in performance. This is intuitive as words without valid morphemes learn noisy embeddings when false morphemes are identified and used to enrich their representation. This is seen in the performance gap between FastText and SkipGram on semantic analogies whereby FastText’s large number indiscriminate subwords degrades the quality of the final embedding. This degradation is mitigated by utilizing more-refined morpheme methods such as BytePair Encoding, Unigram-LM, Morfessor, and MorphMine. Overall, embeddings enriched with MorphMine morphemes demonstrate superior syntactic performance to all baselines while demonstrating comparable semantic performance to SkipGram. This supports our intuition that utilizing more subwords is useful, but only when they are of high quality; indiscriminately generating all enumerations of subword degrades quality.