Attention over Parameters for Dialogue Systems
Figure 3: Results for the Persona-Chat dataset.
['[BOLD] Model', '[BOLD] Ppl.', '[BOLD] F1', '[BOLD] C', '[BOLD] BLEU']
1) AoP and AoR perform consistently better then other baselines which shows the effectiveness of combining parameters by using the correct prior V; 2) AoP performs consistently, but marginally, better than AoR, with the advantage of an algorithmic faster inference; 3) Using Oracle (AoP+O) gives the highest performance in all the measures, which shows the performance upper-bound for AoP. Hence, the performance gap when not using oracle attention is most likely due to the error in attention α (i.e., 2% error rate). the model performance decreases, which confirms that good inductive bias is important for learning how to select and combine different parameters (experts). In general, the difference in performance among different models is marginal except for the Consistency score; thus, we can conclude that all the models can learn this skill reasonably well. Consistently with the previous results, when LV is removed from the optimization, the models’ performance decreases. By adding the layer recursion, both models are able to consistently improve all the evaluated measures, in both Persona-Chat and the Task-Oriented tasks. Especially AoP, which achieves better performance than Oracle (i.e. single layer) in the SQL accuracy, and a consistently better performance in the Persona-Chat evaluation.