Subword-Level Language Identification for Intra-Word Code-Switching
Table 3: Segmentation and LID test results for mixed words only.
['[EMPTY]', 'DE–TR Segmentation', 'DE–TR Segmentation', 'DE–TR Segmentation', 'DE–TR Tagging', 'DE–TR Tagging', 'DE–TR Tagging', 'DE–TR Char', 'ES–WIX Segmentation', 'ES–WIX Segmentation', 'ES–WIX Segmentation', 'ES–WIX Tagging', 'ES–WIX Tagging', 'ES–WIX Tagging', 'ES–WIX Char']
Here, we can see that: (i) Our SegRNN model achieves the best performance for segmentation. Differences to the other approaches are ≥10%, showing clearly why these models are good for the task when the number of words belonging to two languages is high. (ii) The pipeline BiLSTM models work best for tagging of the DE–TR data with a slight margin, but underperform on the ES– WIX dataset as compared to the SegRNN models. (iii) Both CRFTag models achieve very low results for both segmentation and tagging. (iv) CharBiLSTM performs better than the CRFTag models on both tasks, but is worse than all other approaches in our experiments.