Distributional Discrepancy: A Metric for Unconditional Text Generation
Table 4: Kendall’s Tau rank correlation on a real corpus. 5LSTM and 5GPT-2 denote the correlations of five LSTM-based and five GPT-2 based generators, respectively.
['Metric', '5LSTM', '5GPT-2']
The rank achieved by the DD matches the gold-standard rank perfectly across the two architectures. For comparison of the previous metrics, Kendall’s Tau co-efficiency is computed. Our novel metric still achieves the best performance. Although the LM score works considerably well for GPT-2, it fails to discriminate the generators when an LSTM architecture is adapted.