Pretrained Transformers Improve Out-of-Distribution Robustness
Table 8: Out-of-distribution detection AUROC scores for various NLP models using the maximum softmax probability anomaly score. An AUROC score of 50% is random chance, while 100% is perfect.
['Din', 'Dtestout', 'BoW', 'Avg w2v', 'Avg GloVe', 'LSTM w2v', 'LSTM GloVe', 'ConvNet w2v', 'ConvNet GloVe', 'DistilBERT', 'BERT Base', 'BERT Large', 'RoBERTa']
We also report the Area Under the Receiver Operating Characteristic ( AUROC) Hendrycks and Gimpel The AUROC is the probability that an OOD example receives a higher anomaly score than an in-distribution example, viz., P(−maxyp(y∣xout)>−maxyp(y∣xin)). A flawless AUROC is 100% while 50% is random chance.