Decoding Decoders: Finding Optimal Representation Spaces For Unsupervised Similarity Tasks
Table 6: Performance of different architectures and sentence representations on supervised transfer tasks. On each task, the highest performing setup for each encoder type is highlighted in bold and the highest performing setup overall is underlined. All reported values indicate test accuracy on the task, except for SICK-R where we report the Pearson correlation with human-provided scores.
['[BOLD] Encoder', '[BOLD] Decoder', '[BOLD] MR', '[BOLD] CR', '[BOLD] MPQA', '[BOLD] SUBJ', '[BOLD] SST', '[BOLD] TREC', '[BOLD] MRPC', '[BOLD] SICK-R', '[BOLD] SICK-E']
While the strong performance of \glsrnn-\glsrnn-mean relative to \glsrnn-\glsrnn is consistent with our theory, exploring why it is able to outperform \glsrnn-concat experimentally on \glssts tasks is left to future work.