CoCon: A Self-Supervised Approach for Controlled Text Generation
Table 2: Content similarity and quality of generated content-conditioned samples. BLEU, NIST and METEOR values are reported in scale of (×10−2).
['Model', 'BLEU-4', 'NIST-4', 'METEOR', 'Perplexity', 'Dist-1', 'Dist-2', 'Dist-3']
, all the CoCon variants can incorporate the content of c in the generated text better than an unconditioned plain GPT-2 LM. While the CoCon ablated variants appear to be better at incorporating c’s content, it comes at a high cost of text quality for the case of omitted Lcycle and Lnull. If Lcycle were removed, CoCon would train only on prompt text p and content input c segments that were sampled from the same parent x, which explains why the quality of its outputs drops during test time when prompt text p and content input c are from different sources.