SJTU-NLP at SemEval-2018 Task 9: Neural Hypernym Discovery with Term Embeddings
Table 2: Gold standard evaluation on general-purpose subtask.
['Embedding', 'Model', 'MAP', 'MRR', 'P@1', 'P@3', 'P@5', 'P 15']
All the neural models outperform term embedding averaging in terms of all the metrics. This result indicates simply averaging the embedding of words in a phrase is not an appropriate solution to represent a phrase. Convolution or recurrent gated mechanisms in either CNN-based (CNN, RCNN) or RNN (GRU, LSTM) based neural networks could essentially be helpful of modeling the semantic connections between words in a phrase, and guide the networks to discover the hypernym relationships. We also observe CNN-based network performance is better than RNN-based, which indicates local features between words could be more important than long-term dependency in this task where the term length is up to trigrams.