Unlocking the Power of Deep PICO Extraction: Step-wise Medical NER Identification
Table 2: Comparison of PICO classification models.
['[EMPTY]', 'P Precision', 'P Recall', 'P F1', 'I/C Precision', 'I/C Recall', 'I/C F1', 'O Precision', 'O Recall', 'O F1']
The 500 papers for the PICO classification model are downloaded from PubMed sorted by time. Two experts are invited to label the same batch of papers at the same time independently. If there is a disagreement, it will be confirmed by the third expert. We finally get our dataset and divide it into train set and validation set with a 7:3 ratio. Yuan et al. We also compare with them. Our model achieves the best performance on the P label sentences classification comparing with the state-of-the-art models as well as some indicators of I/C and O labels, which proves the effectiveness of our model. In addition to the performance comparison, we also analyze the negative samples. Statistics show that in some cases, due to the frequent occurrence of some typical words, the sentence of P label is difficult to distinguish from I/C, the same as O and I/C. In addition, in some sentences, the part of one sentence belongs to P label, however, the other part belongs to O label. This increases the difficulty of classification. We compare with the results of two recent related papers, and we can clearly see that our results in most indicators are better.