Injecting Hierarchy with U-Net Transformers
Table 2: Comparison of U-Net Transformer against different baselines on the PersonaChat dataset. Model run until lowest validation loss achieved. Per-word perplexity evaluated on held-out test dataset.
['Architecture', 'Lowest Validation Cross Entropy', 'Test Perplexity']
In addition to outperforming in the movie domain, the proposed U-Net Transformer outperforms the vanilla Transformer and all other baselines on the PersonaChat dataset. This dataset consists of everyday conversations with long dialogue histories, and serves to measure how our model performs for larger dialogue histories.