Real or Fake? Learning to Discriminate Machine from Human Generated Text
Table 9: Number of parameters in millions for the scoring functions. The computational cost is directly related to the number of parameters in other layers than the input embedding layer (second row).
['[EMPTY]', '[BOLD] EBM Functions Linear', '[BOLD] EBM Functions BiLSTM', '[BOLD] EBM Functions BiLSTM Big', '[BOLD] EBM Functions UniTransf', '[BOLD] EBM Functions BiTransf']
Training is stopped after processing 2.5M samples without any improvement on the validation set. We use data-parallel synchronous multi-GPU training with up to 8 nodes, each with 8 Nvidia V100 GPUs. The Wikipedia dataset has lower accuracy because the EBM overfits to this smaller dataset.