Is this Dialogue Coherent? Learning from Dialogue Acts and Entities
Table 1: Train, development and test data size for response selection for both Internal and External Swap.
['[EMPTY]', '[BOLD] Train', '[BOLD] Dev', '[BOLD] Test']
For each dialogue, we randomly choose ten insertion points. Each insertion point is composed by a context (dialogue history up to that point) and the original turn following that context (regarded as positive). In order to have 10 next turn candidates, for each insertion point 9 adversarial turns (regarded as negatives) are then randomly selected either from subsequent parts of the dialogue, i.e. Internal Swap (IS), or from other dialogues, i.e. External Swap (ES), within the same data subset, so that for example external adversarial turns for training are only taken from other source dialogues in the training set.