Distilling Task-Specific Knowledge from BERT intoSimple Neural Networks
Table 1: Test results on different datasets. The BiLSTM results reported by other papers are drawn from Zhou et al. (2016),† Wang et al. (2017),‡ and Williams et al. (2017).∗ All of our test results are obtained from the GLUE benchmark website.
['[BOLD] #', 'Model', 'SST-2', 'QQP', 'MNLI-m', 'MNLI-mm']
For QQP, we report both F1 and accuracy, since the dataset is slightly unbalanced. Following GLUE, we report the average score of each model on the datasets.