A Silver Standard Corpus of Human Phenotype-Gene Relations
Table 2: The Known and Unknown number of relations selected, the number of true positives, false negatives, false positives and true negatives, and the evaluation metrics for the Known relations.
['[BOLD] Relations  [BOLD] Known', '[BOLD] Relations  [BOLD] Unknown', '[BOLD] Marked Relations  [BOLD] True Positive', '[BOLD] Marked Relations  [BOLD] False Negative', '[BOLD] Marked Relations  [BOLD] False Positive', '[BOLD] Marked Relations  [BOLD] True Negative', '[BOLD] Metrics  [BOLD] Precision', '[BOLD] Metrics  [BOLD] Recall', '[BOLD] Metrics  [BOLD] F-Measure']
The inter-curator agreement score, calculated from a total of 20 relations classified by eight curators, was 87.58%. Besides the fact that there were a few incorrectly extracted relations due to errors in the NER phase, that were discarded, the inter-curator agreement is not higher due to the complexity of the sentences where the relations between entities were identified. Even with highly knowledgeable curators in the fields of Biology and Biochemistry, most of them expressed difficulties in deciding which mark to choose on complex sentences that did not necessarily imply a relation between the identified entities (Example 3).