Posterior Calibrated Training on Sentence Classification Tasks
Table 5: Hyper-parameters for PosCal training across tasks : the number of updating empirical probabilities per epoch u and weight value λ for LCal. We tune them using the validation set.
['[BOLD] xSLUE GYAFC', '[ITALIC] u 5', '[ITALIC] λ 0.6', '[BOLD] GLUE CoLA', '[ITALIC] u 5', '[ITALIC] λ 0.2']
All models are trained with equal hyper-parameters:learning rate 2e-5, and BERT model size BERTBASE. Also, we set up an early stopping rule for train: we track the validation loss for every 50 steps and then halt to train if current validation loss is bigger than the averaged 10 prior validation losses (i.e., patience 10). For L1, we use the regularization weight value 1-e8. For PosCal, we set up another weight value λ for LCal, and the number of updating empirical probability per epoch (u). We tune these two hyper-parameters per each task. As a baseline of post-calibration method, we also report ECE with a temperature scaling guo2017calibration, which is current state-of-the-art method.