Unlocking the Power of Deep PICO Extraction: Step-wise Medical NER Identification
Table 4: Result of Mapping Model with different method.
['Model', 'Recall P', 'Recall O']
The mapping model is used to perform further processing of the DNER model and obtain the disease entity classification results, so that the clinical practitioners can quickly get the key evidence with less time. Several clinical researchers read series of medical papers of risk models and summarize a set of linguistic rules such as “… risk of  ”. If a sentence matches this rule, the entity that appears in the later part is extracted and classified as “O”. 100 papers are labeled by the clinical researchers following the same protocol with the PICO classification labeling work. For the mapping model, since some sentences may have key elements of P and O at the same time, classification is difficult and has a great impact on the results of DNER. The mapping model uses a set of statistical rules and language rules to alleviate this problem and improve the overall performance. The precision actually is low due to following issues: (1) there are still errors from the PICO classification model and the DNER model which have not been well addressed, and (2) there’s lack of public standard labeling strategies. Note that the manual work of labeling disease entities with PICO elements is a non-trivial work for most of the clinical researchers even we restrict the papers with risk model type. During the experiment, we observe that a good deal of labels done by the first two researchers are mismatched. That also increase the work of the third researchers to further do a decision. By considering this situation and less number of labeled samples, we finally decide to use rule-based method in the mapping model. Specifically for risk model disease entity extraction and classification, this is the first try to do such experiments and can serve as a benchmark for future research.