A Discrete Hard EM Approach for Weakly Supervised Question Answering
Table 6: (Left) Ablations with varying values of τ on TriviaQA. (Middle) Ablations with varying values of τ on DROPnum with BERT. (Right) Final τ chosen for the main results on each dataset. Note that for DROPnum with QANet and WikiSQL, we just report the number without annealing.
['[ITALIC] τ', 'TriviaQA\xa0F1']
To prevent the model to be optimized on early decision of the model, we perform annealing: at training step t, the model optimizes on MML objective with a probability of min(t/τ,1) and otherwise use our objective, where τ is a hyperparameter. We observe that the performance is improved by annealing while not being sensitive to the hyperparameter τ. Note that for DROPnum with QANet and WikiSQL, we do not ablate with varying τ and just report the number without annealing.