Real or Fake? Learning to Discriminate Machine from Human Generated Text
Table 4: Cross-corpora generalization accuracy using TransfBig generator and UniTransf energy function (except for the last row which used a bidirectional transformer). Each row specifies the corpora used at training time, Ctrain. Each column shows the corpus used at test time, Ctest.
['train corpora', 'test corpora Books', 'test corpora CCNews', 'test corpora Wiki']
We observe that models generalize less well across corpora; for instance, when testing on Wikitext an energy function trained with either Books or CCNews, the accuracy is 59.1% and 65.5%, respectively. However, training on the union of two of the corpora gives a large benefit over training on just one or the other when testing on the third.