Empower Sequence Labeling with Task-Aware Neural Language Model
Table 10: Effect of language model and highway
['State Size', 'Model', 'F1score±std', 'Recall±std', 'Precision±std']
The first keeps highway units, but optimizes JCRF alone; the second jointly optimizes JCRF and JLM, but without highway units. This observation accords with previous comparison between TagLM (index 10) and TagLM (index 11) on the CoNLL03 NER dataset. We conjecture that it is because the NER task and the language model is not strongly related to each other. In summary, our proposed co-training strategy is effective and introducing the highway layers is necessary.