Word Ordering Without Syntax
Table 1: BLEU score comparison on the PTB test set. Results from previous works (for ZGen) are those provided by the respective authors, except for the Words task. The final number in the model identifier is the beam size, +gw indicates additional Gigaword data. Models marked with +pos are provided with a POS dictionary derived from the PTB training set.
['Model', 'Words', 'Words+BNPs']
On the Words+BNPs task the NGram-64 model scores nearly 5 points higher than the syntax-based model ZGen-64. The LSTM-64 then surpasses NGram-64 by more than 5 BLEU points. Differences on the Words task are smaller, but show a similar pattern. Incorporating Gigaword further increases the result another 2 points. Notably, the NGram model outperforms the combined result of ZGen-64+lm+gw+pos from \newciteLiuAndZhang2015_empirical_comparison, which uses a 4-gram model trained on Gigaword. We believe this is because the combined ZGen model incorporates the n-gram scores as discretized indicator features instead of using the probability directly.