Cloze-driven Pretraining of Self-attention Networks
Table 3: CoNLL-2003 Named Entity Recognition results. Test result was evaluated on parameter set with the best dev F1.
['[BOLD] Model', '[BOLD] dev F1', '[BOLD] test F1']
Both of our stacking methods outperform the previous state of the art, but fine tuning gives the biggest gain.