Unsupervised Transfer Learning via BERT Neuron Selection
Table 2: Single source unsupervised transfer learning on the multi-domain sentiment analysis dataset.
['S —> T', 'CDB ', 'CMD ', 'PBLM ', 'PE ', '[ITALIC] K', 'BERT 1024', 'SingleTSNS (ours) 100', 'SingleTSNS (ours) 300', 'SingleTSNS (ours) 500', 'SingleTSNS (ours) 700', 'SingleTSNS (ours) 1024']
The selected neurons by SingleTSNS outperform the previous state-of-the-art in 5 out of 12 pairs of domains for the MD dataset. It also outperforms the accuracy of the average of all 12 transferring tasks by 1.1% when compared to the literature and 4.1% when compared to the last layer BERT. This improvement indicates that the selected neurons carry more generalized information regarding the sentiment analysis task. Also, based on the fingerprints of the sentiment analysis datasets in Fig. Additionally, tasks or domains that have similar patterns in the activated layers of their task-specific fingerprints are the best candidates for transferring among them. For example, the best domain for transferring to Books is DVD and vice versa. Similarly, the best domain for transferring to Kitchen is Electronics and vice versa.