Joint Learning of Sentence Embeddings for Relevance and Entailment
Figure 8: Model (question-level) accuracy on the test split of the MCTest task, using the evidence weighing scheme. The first column shows accuracy on a train split joined across both datasets. ∗ The model with top MC-500 test set result (across 16 runs) that convincingly dominates HABCNN-TE in the one and all classes and illustrates that the issue of reporting evaluation spread is not just theoretical. 5/16 of the models have MC-160 all accuracy >0.631.
['Model', 'joint all (train)', 'MC-160 one', 'multi', 'all', 'MC-500 one', 'multi', 'all']
For the MCTest dataset, Fig. We see that averaging-based models are surprisingly effective on this task, and in particular on the MC-500 dataset it can beat even the best so far reported model of HABCNN-TE. Our proposed transfer model is statistically equivalent to the best model on both datasets (furthermore, previous work did not include confidence intervals, even though their models should also be stochastically initialized).