EMNLP’18 Joint Multilingual Supervision for Cross-lingual Entity Linking
Table 7: Linking accuracy of the zero-shot (Z-S) approach on different datasets. Zero-shot (w/ prior) is close to SoTA for datasets like TAC15-Test, but performance drops in the more realistic setting of zero-shot (w/o prior) (§6.1) on all datasets, indicating most of the performance can be attributed to the presence of prior probabilities. The slight drop in McN-Test is due to trivial mentions, which only have a single candidate.
['[BOLD] Dataset →  [BOLD] Approach ↓', 'TAC15-Test (es)', 'TAC15-Test (zh)', 'TH-Test (avg)', 'McN-Test (avg)']
To evaluate the effectiveness of the zero-shot XEL approach, we perform zero-shot XEL using Xelms on all datasets. Note that zero-shot XEL (with prior) is close to SoTA (\newciteSil2018) on TAC15-Test, which also uses the prior for zero-shot XEL. However, for zero-shot XEL (without prior) performance drops by more than 20% for TAC15-Test, 2.4% for TH-Test and by 2.1% for McN-Test. This indicates that zero-shot XEL is not effective in a realistic zero-shot setting (i.e., when the prior is unavailable for inference).