Beyond Leaderboards: A survey of methods for revealingweaknesses in Natural Language Inference data and models
Figure 1: Number of premise-hypothesis pairs in an RTE dataset following lexical patterns, spuriously skewed towards Entailment [mccoy2019right].
['[BOLD] Paragraph:  [ITALIC] “[…] The past record was held by  John Elway , who led the Broncos to victory in Super Bowl XXXIII at age 38 and is currently Denver’s Executive Vice President of Football Operations and General Manager.  Quarterback Jeff Dean had jersey number 37 in Champ Bowl XXXIV.” Question:  [ITALIC] “What is the name of the quarterback who was 38 in Super Bowl XXXIII?”Original model prediction:  John Elway Model prediction after inserting a distracting sentence:  Jeff Dean']
The exploitation of those correlations in order to produce the expected prediction is known as the “Clever Hans Effect”, named after a horse that was believed to perform arithmetic tasks but was shown to react to subtle body language cues of the asking person [Johnson1911].