Natural Language Processing (almost) from Scratch
Table 8: Comparison in generalization performance of benchmark NLP systems with our (NN) approach on POS, chunking, NER and SRL tasks. We report results with both the word-level log-likelihood (WLL) and the sentence-level log-likelihood (SLL). We report with (LMn) performance of the networks trained from the language model embeddings (Table 7). Generalization performance is reported in per-word accuracy (PWA) for POS and F1 score for other tasks.
['[BOLD] Approach', '[BOLD] POS', '[BOLD] CHUNK', '[BOLD] NER', '[BOLD] SRL']
It is worth mentioning the larger language model led to even better performance. This suggests that we could still take advantage of even bigger unlabeled datasets.