A Batch Normalized Inference Network Keeps the KL Vanishing Away
Table 4: Comparison on dialogue generation.
['[BOLD] Model', '[BOLD] CVAE', '[BOLD] CVAE (BOW)', '[BOLD] BN-VAE']
Automatic evaluation: From the KL values, we find that CVAE suffers from posterior collapse while CVAE (BOW) and our approach avoid it effectively. For BLEU-4, we observe the same phenomenon in the previous work liu2019cyclical; zhao2017learning that CVAE is slightly better than the others. This is because CVAE tends to generate the most likely and safe responses repeatedly with the collapsed posterior. As for precision, these three models do not differ much. However, CVAE (BOW) and our BN-VAE outperform CVAE in recall with a large margin. This indicates that BN-VAE can also produce diverse responses with good quality like CVAE (BOW).