Overcoming Practical Issues of Deep Active Learning and its Applications on Named Entity Recognition
Table 4: Performance sensitivity to the number of clusters (J). Notice that the number of total clusters for the word and word + sentence feature is J2 (e.g., 225 for J=15). The micro-F1 scores (%) are the average over all the training set sizes in Figure 5. The range of training set sizes are 40,000–200,000, 40,000–130,000, 40,000–250,000 for CoNLL 2003, NCBI disease, and MedMentions ST19, respectively. The highest F1 scores using different numbers of clusters for each sampling method are highlighted.
['[EMPTY]', 'CoNLL 2003 Val', 'CoNLL 2003 Test', 'NCBI disease Val', 'NCBI disease Test', 'MedMentions ST19 Val', 'MedMentions ST19 Test']
We could see that the micro-F1 scores are very close to each other except in the testing set in NCBI diease dataset. The results suggest that cluster number is a trade-off in EDG. Increase the number of clusters decreases the bias but increases the variance in the error decay estimation.