What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue
Table 2: RSA correlation between speaker/referent embeddings We and token embeddings Wt of the entities’ names, for main entities vs. all entities (right)
['model type', 'main', 'all']
For proper nouns (character names), entity-centric models learn to align the token embeddings with the entity representations (identical to the speaker embeddings). For instance, if the two spaces are topologically similar, the relationship of entities 183 and 335 in the entity library will be analogous to the relationship between the names Joey and Ross in the token space. This reveals that in entity-centric models the space of speaker/referent embeddings is topologically very similar to the space of token embeddings restricted to the entities’ names, and more so than in the biLSTM baseline. We hypothesize that entity-centric models can do the alignment better because referent (and hence speaker) embeddings are closer to the error signal, and thus backpropagation is more effective (this again helps with lower-frequency entities).