Distributional Discrepancy: A Metric for Unconditional Text Generation
Table 5: Kendall’s Tau rank correlation on syntactic data. 10Gs denotes the Tau value when all 10 generators are evaluated together. 5LSTM and 5GPT-2 denote the correlations of five LSTM-based and five GPT-2-based generators, respectively.
['Metric', '5LSTM', '5GPT-2', '10Gs']
We evaluate all 10 generators based on their DD. The rank achieved by the DD matches the gold-standard order perfectly across the two models. It is not necessary to adjust the generators’ softmax temperature. This shows that DD is powerful and efficient.