Adaptive Name Entity Recognition under Highly Unbalanced Data
Table 4: Comparing our methods to baseline based on Bi-LSTM-CRF and BERT-FeedForward on F1 metric. The bold value indicates for the best result. The first version Our Method4 is modified from Bi-LSTM-CRF and the other based on BERT-FeedForward.
['[BOLD] Method', '[BOLD] Bi-LSTM-CRF', '[BOLD] BERT - FF', '[BOLD] Double Bi-LSTM-CRF', '[BOLD] Double BERT - FF', '[BOLD] Our Method ', '[BOLD] Our Method']
The probability for the sequence y is given by: p(y|X)=exp(S(X,y)∑y′∈exp(S(X,y′)) (4) The objective function is the maximum likelihood of the probability distribution denoted as: lnp(y|X)=S(X,y)−ln∑y′∈yexp(S(X,y′)) (5) The maximum likelihood of the probability of predictions are maximized during training and the final tag y∗ is given by: y∗=argmaxy′∈yS(X,y) (6) It has been shown that combing CRFs with Bi-LSTM for modelling the tagging task improves the tagging accuracy in general by effectively incorporating several dependencies across the output labels. In order to mitigate this issue, we tried relabeling the samples and training two separate models. For the first model, we mask the Weak class to “Other” and train it to predict tags for Strong class and “other””. Similarly for the second model, the Strong classes are masked to “other” and a new model is trained this relabelled data. It is evident that this relabeling method does not improve the results and thus motivates to follow a novel method of classification using RNN-CNN followed by NER prediction. We also compared the derived results to four other baselines include original Bi-LSTM-CRF, BERT- Feedforward and Double version of each architecture. In Double version of each architecture, we train one model for weak classes, one model for strong classes and finally merge two labels to form the final prediction using a simple merging schedule. Sentence type classification is not used in this version. In the result table, the bold value denotes for the best value cross all methods. For each entity category, we compute the F1 score and using global F1, weighted average F1 and macro-average of F1 for all classes by the following formula: