What do Entity-Centric Models Learn? Insights from Entity Linking in Multi-Party Dialogue
Table 3: Average cosine similarity of mentions with the same referent.
['biLSTM  [BOLD] h [ITALIC] i', 'EntLib  [BOLD] h [ITALIC] i', 'EntLib  [BOLD] q [ITALIC] i', 'EntNet  [BOLD] h [ITALIC] i', 'EntNet  [BOLD] q [ITALIC] i']
We show this effect in two ways. First, we compare the average mean similarity s of mention (13) The hidden layer thus is representing other information than referent-specific knowledge, and the query layer can be seen as extracting referent-specific information from the hidden layer. Second, we compared the models to variants where the cosine-similarity comparison is replaced by an ordinary dot-product transformation, which converts the querying mechanism into a simple further layer. These variants performed almost as well on the reference task, albeit with a slight but consistent edge for the models using cosine similarity.