Construction of the Literature Graph in Semantic Scholar
Table 2: Document-level evaluation of three approaches in two scientific areas: computer science (CS) and biomedical (Bio).
['Approach', 'CS prec.', 'CS yield', 'Bio prec.', 'Bio yield']
In both domains, the statistical approach gives the highest precision and the lowest yield. The hybrid approach consistently gives the highest yield, but sacrifices precision. The TagMe off-the-shelf library used for the CS domain gives surprisingly good results, with precision within 1 point from the statistical models. However, the MetaMap Lite off-the-shelf library we used for the biomedical domain suffered a huge loss in precision. Our error analysis showed that each of the approaches is able to predict entities not predicted by the other approaches so we decided to pool their outputs in our deployed system, which gives significantly higher yield than any individual approach while maintaining reasonably high precision.