SHOMA at Parseme Shared Task on Automatic Identification of VMWEs: Neural Multiword Expression Tagging with High Generalisation
Table 4: The effects of labelling format.
['[EMPTY]', 'Model', 'Token-based P', 'Token-based R', 'Token-based F1', 'MWE-based P', 'MWE-based R', 'MWE-based F1']
We compare the performance with the results of our proposed IOB-like labelling format where a filtering is applied when the beginning-continuation structure is not satisfied. Our technique considers any singular I as a mistake of the model. The results are reported when applying ConvNet+LSTM approach on the test data. We show the results on a sample of selected languages to analyse the possible effects on model performance. Performing no filtering increases the recall while adversely affecting precision in all cases. Regarding MWE-based evaluation, the recall scores would not change as is expected, but the considerable drop in precision causes a significant decrease in MWE-based F1 scores.