Evaluation Benchmarks and Learning Criteriafor Discourse-Aware Sentence Representations
Table 4: Accuracies with baseline encoder on Discourse Coherence task, with or without a hidden layer in the classifier.
['Baseline w/o hidden layer', '52.0']
In all DiscoEval tasks except DC, we use no hidden layer in the neural architectures, following the example of SentEval. However, some tasks are unsolvable with this simple architecture. In particular, the DC tasks have low accuracies with all models unless a hidden layer is used. This shows that DC requires more complex comparison and inference among input sentences. Our human evaluation below on DC also shows that human accuracies exceed those of the classifier based on sentence embeddings by a large margin.