Marve: Measurement Context Extraction from Text
Table 3. Experiment Results - Evaluation Metrics
['[EMPTY]', 'Predicted Negatives', 'Predicted Positives']
Because Marve is rule-based system rather than a generalized statistical model, the recall metric indicates the extent to which measurement language follows concrete rules rather than how well a given model represents the data. As long as the rules generalize and are relatively concise, this type of system is attractive for its speed and transparency. Our recall results imply that a rule-based system such as Marve can identify words and entities related to measurements with high fidelity. While some recall error is expected because language is varied and often misused, these results understate Marve’s recall. Similar to the findings in ClausIE’s experiments, our preliminary analysis suggests that a significant portion of recall error resulted from incorrect dependency parsing rather than the occurrence of undefined patterns.