DQI: Measuring Data Quality in NLP
Table 6: Term 2 for DQIc3, with SIML=0.4
['[BOLD] Split', '[BOLD] e=0.25', '[BOLD] e=0.33', '[BOLD] e=0.5']
The second term utilizes two hyperparameters, the threshold from the first term and the lower bound on the number of sentences that should meet this threshold. The summation term should therefore be low, as it counts the number of sentences that fail to meet the threshold.