SJTU-NLP at SemEval-2018 Task 9: Neural Hypernym Discovery with Term Embeddings
Table 3: Gold standard evaluation on domain-specific subtask. “Embed” is short for “Embedding”.
['Embed', 'Model', 'medical MAP', 'medical MRR', 'medical P@1', 'medical P@3', 'medical P@5', 'medical P 15', 'music MAP', 'music MRR', 'music P@1', 'music P@3', 'music P@5', 'music P 15']
To investigate the performance of neural models on specific domains, we conduct experiments on medical and medicine subtask. All the neural models outperform term embedding averaging in terms of all the metrics and CNN-based network also performs better than RNN-based ones in most of the metrics using word embedding, which verifies our hypothesis in the general-purpose task. Compared with word embedding, the sense embedding shows a much poorer result though they work closely in general-purpose subtask. The reason might be the simple averaging of sense embedding of various domains for a word, which may introduce too much noise and bias the overall sense representation. This also discloses that modeling the sense embedding of specific domains could be quite important for further improvement.