Why do you say they are similar? Interpretable Semantic Textual Similarity
Table 10: First user study: Correlations for non verbalized items, gold standard verbalized items and system verbalized items.
['[EMPTY]', 'No verb', 'SYS verb', 'GS verb']
To measure the results of the first user study, we use the correlation between the scores given by participants and the gold standard STS score. Both correlation measures seem to output very similar values, with higher correlation values for the verbalized scenarios, showing that the explanations are indeed helpful in this task. The verbalizations obtained from the system output are comparable to those of the gold standard, showing that approximate performance might be enough for being helpful in this task. Even if the amount of data points is small, we performed significance tests between the verbalization options using Fisher’s z-transformation for relatedness Finally, the difference between system and gold standard verbalizations is not statistically significant.