Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve
Table 2: Mean gate values when entities and non-entities are correct and incorrect. For entities, the average value of context gates remains the same irrespective of the predicted values. For both entities and non-entities, the word/span gate has a much higher value when the prediction is correct. The word identity itself is the major driver of performance.
['[EMPTY]', '[BOLD] Context', '[BOLD] Span']
Clearly, systems with access to only word identity perform better than those with access to only the context (drop of ∼20 F1 in all the three datasets). For entities, while the context gate value is higher than the word gate value, its average remains the same irrespective of whether the prediction is correct or not. On the other hand, the word gate value drops considerably when the system makes an error. Similarly, the word gate value drops considerably for non-entities as well on error. Surprisingly, the context gate value increases for non-entities when an error is made. These results suggest that systems over-rely on word identity to make their predictions.