CohEval: Benchmarking Coherence Models
Table 4: Results: Accuracies of the models in the Local Discrimination task.
['[BOLD] Model', 'D [ITALIC] w=1,2,3', 'D [ITALIC] w=1', 'D [ITALIC] w=2', 'D [ITALIC] w=3']
A possible reason for the better performance of UnifiedModel could be their loss function. Unlike other models, they use an adaptive pairwise ranking loss which does not penalize the locally coherent sentences. In the local discrimination task, the difference between positive and negative examples is small; most of the cases are locally coherent. UnifiedModelś loss function can capture this.