Self-Supervised Dialogue Learning
Table 5: The experimental results of different dialogue agents at training epoch = {100,200,300}. Each number is averaged over 3 runs, and each run tested on 50 dialogues. The D3Q-SSN denotes the D3Q agent where our proposed SSN replaces the discriminator. The “fixed θD/θSSN” indicates the discriminator/SSN is pre-trained and fixed during the training process. Succ: Success Rate. Reward: Average Reward. Turns: Average Turns.
['[BOLD] Agent', '[BOLD] Planning Steps', '[BOLD] Epoch 100  [BOLD] Succ', '[BOLD] Epoch 100  [BOLD] Reward', '[BOLD] Epoch 100  [BOLD] Turns', '[BOLD] Epoch 200  [BOLD] Succ', '[BOLD] Epoch 200  [BOLD] Reward', '[BOLD] Epoch 200  [BOLD] Turns', '[BOLD] Epoch 300  [BOLD] Succ', '[BOLD] Epoch 300  [BOLD] Reward', '[BOLD] Epoch 300  [BOLD] Turns']
S4SS3SSS0Px4 Results From the results, we can observe that: (1) The D3Q-SSN outperform the D3Q in the most of cases, which shows that our SSN-based discriminator can improve the ability to recognize the high-quality stimulated user experiences. (2) When the planning step increases in D3Q, the performance shows an apparent drop. This is because the discriminator D in the original D3Q agent keeps lots of low-quality stimulated user experiences, which significantly degrade the performance of the D3Q agent. As for our SSN, we can see some performance improvement even when using 10-step planning. This substantially means that our SSN has a better ability to select the good simulated user experiences, especially in the multi-turn dialogue cases.