Unsupervised Transfer Learning via BERT Neuron Selection
Table 3: Single source unsupervised transfer learning to the QQP and SciTail.
['Source', 'Target', 'BERT 1024', '[EMPTY]', 'SingleTSNS (ours) 100', 'SingleTSNS (ours) 500', 'SingleTSNS (ours) 1024', 'Target', 'BERT 1024', '[EMPTY]', 'SingleTSNS (ours) 100', 'SingleTSNS (ours) 500', 'SingleTSNS (ours) 1024']
Fig. Furthermore, when comparing to selecting neurons in the last layer of BERT as a baseline, the accuracy increases significantly. Based on our best knowledge, there is not any other reference for completely unsupervised transfer learning to QQP and SciTail datasets without fine-tuning the language transformer.