Topological Sort for Sentence Ordering
Table 4: Sentence Displacement Analysis for all the datasets. (Win=Window size; % Miss=% mismatch)
['[BOLD] Model', '[BOLD] Win=1  [BOLD] NIPS', '[BOLD] Win=2  [BOLD] NIPS', '[BOLD] Win=3  [BOLD] NIPS', '[BOLD] % Miss  [BOLD] NIPS', '[BOLD] Win=1  [BOLD] SIND', '[BOLD] Win=2  [BOLD] SIND', '[BOLD] Win=3  [BOLD] SIND', '[BOLD] % Miss  [BOLD] SIND']
Displacement of sentences in predicted orders is measured by calculating the percentage of sentences whose predicted location is within 1, 2 or 3 positions (in either direction) from their original location. A higher percentage indicates less displacement of sentences. Observe that as window size reduces, the difference between B-TSort and B-AON percentages increases. This implies that displacement of sentences is higher in B-AON despite taking the whole document into account. Note that the AON model generates the order and hence need not generate positions for all the sentences in the input. We calculate the percentage of mismatches between the length of the input document and the generated order. The AON model also produces an overall mismatch of 0.84 % on AAN documents while producing a mismatch of 5.17% on longer AAN documents. Similarly, the B-AON model has an overall mismatch of 3.48% for NIPS dataset, and 33.33% mismatch for longer documents. This problem does not arise in our design of the task as it does not have to stochastically generate orders.