Discourse Coherence in the Wild: A Dataset, Evaluation and Methods
Table 6: Score prediction results on test.
['System', 'Spearman  [ITALIC] ρ Yahoo', 'Spearman  [ITALIC] ρ Clinton', 'Spearman  [ITALIC] ρ Enron', 'Spearman  [ITALIC] ρ Yelp']
A 3-point coherence score might not reflect the range of coherence that actually exists in the data. We can instead present a more fine-grained score prediction task where the gold score is the mean of the three expert rater judgments (low coherence = 1, medium = 2, high = 3). As in the classification task, the neural methods convincingly outperformed all other methods, with ParSeq the top performer in three out of four domains.