Efficient Contextual Representation Learning Without Softmax Layer
Table 4: Performance of models including ELMo-Sub on downstream tasks. We show the average score of ten classification tasks from SentEval and showcase five of them.
['Models', 'SNLI', 'Coref', 'SST-5', 'NER', 'SentEval  [BOLD] Avg', 'SentEval SST-5', 'SentEval SST-2', 'SentEval TREC', 'SentEval MR', 'SentEval SUBJ']
On SentEval, SST-5, and NER, it has similar performance with ELMo-S\textscOneB. However, it lags behind on Coref and has almost breaking performance on SNLI. It even failed to outperform FastText\textsccc on SNLI, a non-contextual model, which we considered baseline. They find that special design has to be made to apply BERT to constituency parsing because of the subword segmentation.