Latent Predictor Networks for Code Generation
Table 4: Results with increasing compression rates with a regular softmax (cf. “Softmax”) and a LPN (cf. “LPN”). Performance values (cf. “Seconds Per Card” block) are computed using one CPU.
['Compression Seconds Per Card', 'Compression Seconds Per Card', '0%', '20%', '40%', '60%', '80%']
S7SS1SSS0Px2 Component Comparison We present ablation results in order to analyze the contribution of each of our modifications. Removing the C2W model (cf. “– C2W” row) yields a small deterioration, as word lookup tables are more susceptible to sparsity. The only exception is in the HS dataset, where lookup tables perform better. We believe that this is because the small size of the training set does not provide enough evidence for the character model to scale to unknown words. Surprisingly, running our model compression code (cf. “– Compress” row) actually yields better results. We obtain the best results with an 80% compression rate (cf. “BLEU Scores” block), while maximising the time each card is processed (cf. “Seconds Per Card” block). This applies when using the regular optimization process with a character softmax (cf. “Softmax” rows), but also when using the LPN (cf. “LPN” rows). We also note that the training speed of LPNs is not significantly lower as marginalization is performed with a dynamic program. Finally, a significant decrease is observed if we remove the pointer networks (cf. “– LPN” row). These improvements also generalize to sequence-to-sequence models (cf. “– Attention” row), as the scores are superior to the sequence-to-sequence benchmark (cf. “Sequence” row).