Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining
Table 5: Change of prediction accuracy when the selected layer indices Ls take different values (%).
['[ITALIC] Ls↓ [BOLD]  Dataset→', '[BOLD] Amazon', '[BOLD] Taxonomy']
We also vary the number of corrupted domain classifiers by changing Ls. In a majority of scenarios, adding more corrupted domain classifiers slightly improve the performance, as it poses strong domain-invariance constraints to deeper layers of transformer encoders in BERT.