Piecewise Latent Variables for Neural Variational Text Processing
Table 3: Ubuntu evaluation using F1 metrics w.r.t. activities and entities. G-VHRED, P-VHRED and H-VHRED all outperform the baseline HRED. G-VHRED performs best w.r.t. activities and H-VHRED performs best w.r.t. entities.
['[BOLD] Model', '[BOLD] Activity', '[BOLD] Entity']
All latent variable models outperform HRED w.r.t. both activities and entities. This strongly suggests that the high-level concepts represented by the latent variables help generate meaningful, goal-directed responses. Furthermore, each type of latent variable appears to help with a different aspects of the generation task. G-VHRED performs best w.r.t. activities (e.g. download, install and so on), which occur frequently in the dataset. This suggests that the Gaussian latent variables learn useful latent representations for frequent actions. On the other hand, H-VHRED performs best w.r.t. entities (e.g. Firefox, GNOME), which are often much rarer and mutually exclusive in the dataset. This suggests that the combination of Gaussian and piecewise latent variables help learn useful representations for entities, which could not be learned by Gaussian latent variables alone. We further conducted a qualitative analysis of the model responses, which supports these conclusions.