Annotating and Extracting Synthesis Processof All-Solid-State Batteries from Scientific Literature
Table 6: F1 scores of rule-based system and ablation test results. Macro-averaged F1 scores were calculated using Condition and Next (ALL).
['[EMPTY]', 'Condition', 'Next', 'ALL']
The high performance with a macro-averaged F1 score of 0.887 shows the effectiveness of the rules. To confirm the effectiveness of the fine-grained types or sub-labels, we compared the F1 score with three settings. In the first setting, we extracted the relations without using material sub-labels (– Material-∗), by applying the rule of M-O to all of the Material types and ignoring the rule of O-M. In the second setting, we extracted relations without using Property sub-labels (– Property-∗), by applying the rule of Po-OM to all of the Property types and ignoring the rule of P-O. The final setting was without either Material or Property sub-labels (– both sub-levels). According to the ablation tests, the F1 scores were improved by 7.8% on Condition and 11.1% on Next when applying the sub-label rules.