Online Segment to Segment Neural Transduction
Table 2: Perplexity on the validation set with 172k sentence-summary pairs.
['Model', 'Configuration', 'Perplexity']
To make further comparison, we experimented with different sizes of hidden units and adding more layers to the baseline encoder-decoder. We can see that the vanilla encoder-decoder tends to get better results by adding more hidden units and stacking more layers. This is due to the limitation of compressing information into a fixed-size vector. It has to use larger vectors and deeper structure in order to memorize more information. By contrast, our model can do well with smaller networks. In fact, even with 1 layer and 128 hidden units, our model works much better than the vanilla encoder-decoder with 3 layers and 256 hidden units per layer.