Hierarchical Character-Word Models for Language Identification
Table 3: F1 scores on the TweetLID language ID task (constrained track), averaged and per language category (including undetermined and ambiguous). The scores for hurtado2014elirf and gamallo2014comparing are as reported in zubiaga2014overview; per-language scores are not available.
['[BOLD] Model', '[BOLD] Avg.  [ITALIC] F1', '[BOLD] eng', '[BOLD] spa', '[BOLD] cat', '[BOLD] eus', '[BOLD] por', '[BOLD] glg', '[BOLD] und', '[BOLD] amb']
Our model achieves the state of the art on this task, surpassing the shared task winner, \newcitehurtado2014elirf. As expected, C2L fails to match the performance of C2V2L, demonstrating that there is value in the hierarchical representations. The performance of the n-gram LM baseline is notably strong, beating eleven out of the twelve submissions to the TweetLID shared task. Note that performance on underrepresented categories such as ’glg’ and ’und’ is much lower than the other categories. The category breakdown is not available for previously published results.