CohEval: Benchmarking Coherence Models
Table 2: Results: Accuracies of the coherence models in the Global Discrimination task.
['[BOLD] Model', '[BOLD] Emb.', '[BOLD] Standard', '[BOLD] Inverse']
We see that UnifiedModel achieves the highest accuracy on the standard order discrimination task and TransModel performs the best on the Inverse order discrimination task. The other three models use entity grid, hence they lose the sentential structure of the document. UnifiedModel and TransModel both capture sentence grammar and discourse relations. Their performance on the global discrimination task corroborates this.