Definition Modeling: Learning to define word embeddings in natural language
Table 4: Perplexity evaluated on dictionary entries in the test set (lower is better).
['Model', '#Params', 'Perplexity']
First, we compare our different methods for utilizing the word being defined within the models. We see that the gated update (S+G) improves the performance of the Seed, while the other architectures (S+I and S+H) do not. The results are consistent with our hypothesis that the word being defined is more relevant to some words in the definition than to others, and the gate update can identify this.