Evaluating the word-expert approach for Named-Entity Disambiguation
Table 7: Ambiguity, according to dictionaries, of target strings in the development dataset: top six rows show figures for the LNRM and HEUR dictionaries; last two rows focus on HEUR only, after discarding entities that lack training examples, according to the LEX and SENSE methods (see Section 5).
['[EMPTY]', '[ITALIC] Unique  [ITALIC] Strings', '[ITALIC] No Entities  [ITALIC] (NIL [ITALIC] s)', '[ITALIC] Single Entity  [ITALIC] (Monosemous)', '[ITALIC] Multiple Entities  [ITALIC] (Polysemous)', '[ITALIC] Multiple Entities  [ITALIC] Ambiguity']
We now consider polysemy and synonymy in our dictionaries. For instance, the dev-set comprises 1,162 target strings, of which LNRM has no suggestions for 111 (94 with HEUR). Among the remaining strings, 331 yield a unique suggestion from Wikipedia (331 with HEUR). The average ambiguity for the 765 strings with multiple candidates is 86.14 (only 22.26 with HEUR). The distribution of the ambiguity between news and web datasets is distinct, 50.90 and 103.24, respectively. Thus, HEUR covers more strings and has fewer monosemous entries yet lower polysemy than LNRM. Thus, HEUR retains more references from Wikipedia and simultaneously substantially reduces ambiguity. Although we don’t tabulate figures for other dictionaries, we note here that FUZZ has even higher polysemy than LNRM. The lower half of the table focuses on the HEUR dictionary in combination with the LEX and SENSE strategies for gathering training instances. As expected, resulting ambiguities are lower than for full dictionaries, with more cases of zero or one candidates and fewer strings mapping to multiple entities (also with lower ambiguities); the decrease is smaller for the more conservative filtering strategy (SENSE), which is again consistent with our expectations.