A Hybrid Retrieval-Generation Neural Conversation Model
Table 4. The hyper-parameter settings in the generation-based baselines and the generation module in the proposed hybrid neural conversation model. These settings are the optimized settings tuned with the validation data.
['Models', 'Seq2Seq', 'Seq2Seq-Facts']
Parameter Settings Hyper-parameters are tuned with the validation data. For the hyper-parameter settings in the hybrid ranking module, we set the window size of the convolution and pooling kernels as (6,6). The number of convolution kernels is 64. The dropout rate is set to 0.5. The margin in the pairwise-ranking hinge loss is 1.0. The distant supervision signals and the number of positive samples per context in the hybrid ranking module are tuned with validation data. The used distant supervision signal is BLEU-1 and we treat top 3 response candidates ranked by BLEU-1 as positive samples. All models are trained on a single Nvidia Titan X GPU by stochastic gradient descent with Adam (DBLP: journals/corr/KingmaB14) algorithm. The initial learning rate is 0.0001. The parameters of Adam, β1 and β2, are 0.9 and 0.999 respectively. The batch size is 500. The maximum conversation context/ response length is 30.