To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks
Table 4: Comparison of BERT- cross-sentence embedding methods on dev. sets of sentence pair tasks.
['[EMPTY]', 'SICK-E', 'SICK-R', 'STS-B', 'MRPC']
The latter leads to a drop in performance, which shows that the BERT representations encode cross-sentence relationships and are therefore particularly well-suited for sentence pair tasks.