Noise-robust Named Entity Understanding for Virtual Assistants
Table 4: EL F1 improvements brought by (a) MLP similarity, (b) utterance embedding, (c) log-scale popularity and (d) relation features.
['[EMPTY]', '[BOLD] (b) + Utterance Embedding', '[BOLD] (c) + Log-scale Popularity']
We show the difference in mean F1 scores on the development sets over 10 runs compared with the baseline. As expected, the MLP similarity significantly improves entity linking performance with its capacity to model text variations, especially on utterances with noisy entity name mentions. Utterance embeddings brought improvements in the music, and media & TV domains. The improvement brought by log-scale popularity feature is most significant for the movie & TV domain, where the popularity distribution has extremely long tails compared to other domains. Relation feature is powerful for disambiguating entities with similar names, and we show significant improvement in EL F1 on the subset of utterances that have related entities.