Inducing brain-relevant bias in natural language processing models
Table 1: GLUE benchmark results for the GLUE development sets. We compare the results of two of our models to the results published by https://github.com/huggingface/pytorch-pretrained-BERT/ for the pretrained BERT model. The model labeled ‘MEG’ is the MEG transfer model described in section 3.4. The model labeled ‘Joint’ is the fully joint model also described in section 3.4. For all but one task, at least one of our two models is marginally better than the pretrained model. These results suggest that fine-tuning does not diminish – and possibly even enhances – the model’s ability to perform NLP tasks.
['Metric', 'Vanilla', 'MEG', 'Joint']
NLP tasks are not harmed by fine-tuning. We run two of our models (the MEG transfer model, and the fully joint model) on the GLUE benchmark These models were chosen because we thought they had the best chance of giving us interesting GLUE results, and they were the only two models we ran GLUE on. Apart from the semantic textual similarity (STS-B) task, all of the other tasks are very slightly improved on the development sets after the model has been fine-tuned on brain activity data. The STS-B task results are very slightly worse than the results for standard BERT. The fine-tuning may or may not be helping the model to perform these NLP tasks, but it clearly does not harm performance in these tasks.