Hierarchical Contextualized Representation for Named Entity Recognition
Table 6: Comparison of different strategies on CoNLL-2003 dataset. ERR is the relative error rate reduction of our model compared to the baseline.
['[EMPTY]', 'Strategy', '[ITALIC] F1', 'ERR']
We further analyze the two hierarchical representations by adopting different strategies. We further conduct experiments to investigate the three compatibility functions used to employ memorized information. Among the three compatibility functions to compute the weight of query word and memorized slots, cosine similarity performs best, while dot-product performs worst. Cosine similarity calculates the inner product of word vectors with unit length, and can further solve the inconsistency between the embeddings and the similarity measurement. Thus, we eventually adopt cosine similarity as the compatibility function.