What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models
Table 4: CPRAG-34 word prediction accuracies (with and without sentence perturbations)
['[EMPTY]', 'Orig', 'Shuf', 'Trunc', 'Shuf + Trunc']
For accuracy at k=1, BERTLARGE soundly outperforms BERTBASE with correct predictions on just over a third of items. When we expand to k=5, the models converge on the same accuracy, identifying the expected completion in approximately half of items. Because commonsense and pragmatic reasoning are non-trivial concepts to pin down, it is worth asking to what extent BERT can achieve this performance based on simpler cues like word identities or n-gram context. To test adequacy of truncated context, we remove all words of the second sentence but the two words preceding the target word (“Trunc”). This gives generally enough syntactic context to identify the part of speech, as well as some sense of semantic category, but removes other information from that second sentence. We also test with both perturbations together (“Shuf + Trunc”). Because different shuffled word orders give rise to different results, for the “Shuf” and “Shuf + Trunc” settings we show mean and standard deviation from 100 runs. One thing that is immediately clear is that the BERT model is indeed making use of information provided by the word order of the first sentence, and by the more distant content of the second sentence, as each of these individual perturbations causes a notable drop in accuracy. It is worth noting, however, that with each perturbation there is a subset of items for which BERT ’s accuracy remains intact. Unsurprisingly, many of these items are those containing particularly distinctive words associated with the target, such as checkmate (chess), touchdown (football), and stone-washed (jeans). This suggests that some of BERT ’s success on these items may be attributable to simpler lexical or n-gram information.