Learning Neural Sequence-to-Sequence Models from Weak Feedback with Bipolar Ramp Loss
Table 3: Answer F1 scores on the NLmaps v2 test set for RAMP and the MRT objective as well as two further objectives, which help crystallize the difference between the two former objectives, averaged over two independent runs. M is the minibatch size. All models are statistically significant from each other at p<0.01, except the pair (3, 4).
['1', 'MLE', '[ITALIC] M', '% F1 57.45', 'Δ']
MRT is not naturally a bipolar objective. It can only discourage wrong parses if the baseline is larger than 0. Investigating the value of the baseline for 10,000 instances shows that in 37% of the cases the baseline is 0, i.e. none of the sampled parses leads to the correct answer. As a result, 37% of the time, wrong parses are ignored rather than discouraged. To explore the importance of always discouraging wrong parses, we introduce the objective MRT neg: it modifies the feedback for parses with a wrong answer to be −1 rather than 0, which resembles the fear output that is discouraged in the RAMP objective. With this change, the MRT objective always behaves in a bipolar manner, irrespective of the baseline’s value. However, MRT neg still falls short of RAMP by 3.1 points in F1 score. This could be because of the different batch sizes as MRT uses a batch size of 1, whereas RAMP employs a batch size of 80. To ensure that the difference between the objectives does not stem from this difference, we run an experiment with RAMP where the batch size is also set to 1, i.e. RAMP m=1. Crucially, it still significantly outperforms MRT. This showcases the importance of using a larger minibatch size, so that an average over several inputs is computed before updating. In fact, its F1 score is on par with the MRT neg objective, which uses the same minibatch size and incorporates bipolar supervision just as Ramp does. However, RAMP m=1 should still be preferred because the RAMP objectives are more efficient than MRT objectives. In the case of MRT, for every training instance S+S′=20 queries need to be executed against the database to obtain an answer and corresponding reward. On the other hand, Ramp has to execute at most the 10 queries of the k-best list K, but often less if both a correct and an incorrect query are found earlier.