A Morpho-Syntactically Informed LSTM-CRF Modelfor Named Entity Recognition
Table 9: Confusion matrix for our best model on the test dataset: the columns represent the true labels and the rows show the predictions.
['[EMPTY]', 'B-ORG', 'I-ORG', 'B-PER', 'I-PER', 'B-LOC', 'I-LOC', 'B-MISC', 'I-MISC', 'O']
In the table, the columns represent the actual expected gold tags, while the rows show the predictions of our model. There are several interesting observations that we can make about this confusion matrix. First, it looks like the biggest problem for the model is with the tag B-PER, which is often confused with the tag O, i.e., Outside. This is probably due to the fact that in Bulgarian the first names sometimes have more than one meaning, which can confuse the model. The same argument holds for the tag B-LOC, which is also often confused with the tag O. Another place for improvements would be to distinguish better between B-LOC and B-ORG, as many places and organization have identical names, or at least the identical first words. The miscellaneous entities such as the names of books or movies can also have names that are identical to those of some organizations. Even more often, Miscellaneous entities could be confused with the Other category as they contain many common Bulgarian words.