Learning Neural Sequence-to-Sequence Models from Weak Feedback with Bipolar Ramp Loss
Table 2: Answer F1 scores on the NLmaps v2 test set for various objectives, averaged over two independent runs. M is the minibatch size. All models are statistically significant from each other at p<0.01, except the pair (2, 4).
['1', 'MLE', '[ITALIC] M', '% F1 57.45', 'Δ']
MRT outperforms the MLE baseline by about 6 percentage points in F1 score. RAMP1 performs worse than MRT, but can still significantly outperform the baseline by 3.05 points in F1 score. RAMP2 performs better than RAMP1, but outperforms MRT only nominally.