USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generation
Table 5: Turn-level correlations between all automatic metrics and the Overall Quality ratings for the Topical-Chat corpus. All values with p>0.05 are italicized.
['Metric', 'Spearman', 'Pearson']
USR shows a strong improvement over all other methods. This strong performance can be attributed to two factors: (1) the ability of MLM and DR to accurately quantify qualities of a generated response without a reference response, and (2) the ability of USR to effectively combine MLM and DR into a better correlated overall metric.