Casting a Wide Net: Robust Extraction of Potentially Idiomatic Expressions
Table 9: PIE extraction performance in terms of precision, recall, and F1-score of the best variant by F1-score of each of the four systems, on the test set. CS indicates case-sensitive string matching. The best score for each metric is in bold.
['[EMPTY]', 'Precision', 'Recall', 'F1-score']
We evaluate the best variant of each system, as determined by F1-score, on the test set. This gives us an indication of whether the system is robust enough, or was overfitted on the development data. On average, the results are lower than the results on the development set. The string-based methods perform clearly worse, with drops of about 4% F1-score for exact and inflectional match, and a large drop of almost 9% F1-score for fuzzy matching. The parser-based method, on the other hand, is more robust, with a small 0.59% increase in F1-score on the test set.