Learning Sentence Embeddings for Coherence Modelling and Beyond
Table 2: Results on the order discrimination and sentence reordering coherence tasks. Our approach trades only a small decrease in performance for improved utility of the sentence embeddings over other approaches, achieving close to or the same as the state-of-the-art.
['[BOLD] Model', '[BOLD] Order discrimination  [BOLD] Accident', '[BOLD] Order discrimination  [BOLD] Earthquake', '[BOLD] Reordering  [BOLD] Acc', '[BOLD] Reordering  [ITALIC] τ']
Order discrimination setup. For order discrimination, we use the Accidents and Earthquakes datasets from Barzilay and Lapata The task is to determine which of a permuted ordering of the sentences and the original ordering is the most coherent (in the original order), for twenty such permutations. Performance is measured with the accuracy with which the permuted sentences are identified. Results. For the order discrimination task, we find that on both datasets, our PPD-based approach only slightly underperforms ATTOrderNet Cui et al. Gong et al. ; Logeswaran et al. On the more difficult sentence reordering task, our approach exhibits performance closer to the state-of-the-art, achieving the same ranking correlation and only slightly lower positional accuracy. Given that the publicly available training set for the reordering task is slightly smaller than that used in previous work, it is possible that more data would allow our approach to achieve even better performance. In the next section we will discuss the real-world value offered by our approach that is largely missing from existing approaches.