Energy-Based Models for Text
Table 8: Generalization in the wild of the discriminator to unconditional generation from various GPT2 models (model size in parentheses, followed by sampling method used). Each row contains the accuracy on the corresponding test set. TF-IDF results are taken from Radford and Wu (2019). Results in parentheses are taken from https://openai.com/blog/gpt-2-1-5b-release/.
['Discriminator → Test setting →', 'TF-IDF∗ in-domain', 'BiT in-domain', 'BiT cross-architecture', 'BiT wild']
In this case, we finetune the discriminator on the training set of each of the datasets, following the same protocol used by the provided TF-IDF baseline. We notice that BiT discriminator has consistently superior performance, with an accuracy greater than 95%. While the discriminator still works much better than a random predictor, it lags behind the simple (in-domain) linear baseline. That suggests that matching the domain of the training set is more important than matching the model complexity.