From Credit Assignment to Entropy Regularization: Two New Algorithms for Neural Sequence Prediction
Table 3: Average validation BLEU of ERAC. As a reference, the average BLEU is 28.1 for MLE. λvar=0 means not using the smoothing technique. β=1 means not using a target network. † indicates excluding extreme values due to divergence.
['[ITALIC] λvar [ITALIC] β', '0.001', '0.01', '0.1', '1']
Based on the MT task, we vary the update speed β of the target critic, and the λvar, which controls the strength of the smoothness regularization. The average validation performances of different hyper-parameter values are summarized in Tab. [leftmargin=1.0em,topsep=0.5em,itemsep=0em] Comparing the two rows of Tab. In fact, removing the smoothing objective often causes the training to diverge, especially when β=0.01 and 1. But interestingly, we find the divergence does not happen if we update the target network a little bit faster (β=0.1) or quite slowly (β=0.001). In addition, even with the smoothing technique, the target network is still necessary. When the target network is not used (β=1), the performance drops below the MLE baseline. However, as long as a target network is employed to ensure the training stability, the specific choice of target network update rate does not matter as much. Empirically, it seems using a slower (β=0.001) update rate yields the best result.