Enriching Knowledge Bases with Counting Quantifiers
Table 2: Performance of CINEX on recognizing counting quantifier mentions, with different architectures and in comparison with the baseline. Highest F1-score per relation in boldface.
['Relation', 'Baseline\xa0', 'Baseline\xa0', 'Baseline\xa0', 'CINEX CRF', 'CINEX CRF', 'CINEX CRF', 'CINEX biLSTM', 'CINEX biLSTM', 'CINEX biLSTM', 'CINEX biLSTM-CRF', 'CINEX biLSTM-CRF', 'CINEX biLSTM-CRF']
the performance results of different architectures wrt. precision, recall and F1-score. As one can see, feature-based CRF models are the most robust sequence labeling approach across relations for this task, although the neural models achieve higher F1-score with 3.3 percentage point difference for containsAdmin. Adding a CRF layer on top of bidirectional LSTM models improves performance across relations, although this architecture still fails to beat the feature-based CRF models in most cases. We conjecture that this is due to neural models being much more prone to overfitting to noisy distantly supervised training data. Still, both feature-based and neural models consistently outperform the baseline by a large margin, in particular wrt. precision.