Using Multi-Sense Vector Embeddings for Reverse Dictionaries
Table 2: Median rank, accuracy @10 and @100 and mean reciprocal rank of single- compared to multi-sense target vectors. The first row is the model architecture proposed by Hill et al. (2016).
['[BOLD] Input Vectors', '[BOLD] Target Vector', '[BOLD] MR ↓', '[BOLD] Acc@10 ↑', '[BOLD] Acc@100 ↑', '[BOLD] MRR ↑']
Although the number of candidates is larger when every target word has multiple candidate target vectors, the separation of the representation of the target words into different vectors according to their senses clearly helps the model to produce a reasonable representation of the input sequence. This effect is independent of whether the input is encoded using single- or multi-sense vectors. It should be noted again that the model does not have access to the true sense during testing and that instead all possible sense vectors are used for ranking. The pre-trained, contextual BERT vectors perform very poorly as target vectors. This might be due to the larger vector size, the more complex representation or the missing or uncommon context. In fact, we found that BERT obtains only 0.009 mean reciprocal rank even if we provide it with the ground truth definitions as contexts to generate the target representations.