DQI: Measuring Data Quality in NLP
Table 23: T4 for DQIc6
['[BOLD] Split-Label', '[BOLD] T4']
The fourth term calculates the standard deviation of sentence length difference between premise and hypothesis across labels. The standard deviation needs to be higher to ensure that there exist samples of varying difference betweeen premise and hypothesis length, and the model is not overfitted towards a fixed length difference. It passes for entailment and neutral label. It fails for contradiction label though both the terms are very close in that case.