Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems
Table A.5: Hyper-parameters used for different models.
['[BOLD] Dataset', '[BOLD] Version', '[BOLD] Model', '[BOLD] Batch size', '[BOLD] Dropout', '[BOLD] Decoder hidden size', '[BOLD] Encoder hidden size', '[BOLD] Context size', '[BOLD] Embedding size', '[BOLD] Word drop', '[BOLD] Sentence drop', '[BOLD] Beam size', '[BOLD] Emotion weight', '[BOLD] Emotion discriminator layer size', '[BOLD] Infersent weight', '[BOLD] Infersent discriminator layer size']
For the models that were trained on the Reddit dataset, a set of properly tuned baseline parameters were non-existent. Thus, to ensure fair comparison, we used a similar approach for baseline and EI hyper-parameter tuning: We explored a combination of values for encoder hidden size (400, 600, 800, 1250), decoder hidden size (400, 600, 800, 1250), context size (1000, 1250), embedding size (300, 400, 500, 600), word drop (0, .25), sentence drop (0, .1, .25), and beam size (1, 5). Learning rate (.0001), dropout (.2) were fixed. Batch size 64 was used. If due to memory limitation the job was not successfully completed, batch size 32 was used. Due to limited computational resources, we were not able to run a grid search on all the aforementioned values. Instead we used combinations of the parameters that heuristically were more viable. To ensure fair comparison, any selected combination was tested for both baseline and EI models. Then, for EI models, we tuned the parameters that were solely relevant to the EI design, such as the weight of emotion and infersent term in the loss function and the size of the added discriminator networks: Emotion weight (25), infersent weight (25K, 50K, 100K), emotion sizes (64, 128, 256), infersent sizes (100, 128, 1000, 2000, 4000).