Analyzing representations
Table 10: Results of retraining Infersent on both SNLI and the Comparisons dataset.
['Epoch', 'Performance (%) Train(Combined)', 'Performance (%) Test(Comp)', 'Performance (%) Test(SNLI)']
We then checked whether InferSent can represent this relational structure without losing the information necessary for SNLI. We started with an untrained network, and then trained on an augmented version of the original training data. Here, examples from the SNLI training set were randomly interleaved with examples from our Comparisons training dataset, otherwise using the same training protocols reported in (Conneau \BOthers. We found that the accuracy obtained this way on the SNLI test set (84.96 %) is comparable to the model trained only on SNLI (84.84 %). Moreover, test accuracy on the Comparisons dataset is close to perfect (99.55 %) and is much higher than the model trained only on SNLI (47.81 %). This establishes that in this case the model has enough capacity to achieve high performance on specially designed edge-cases like the Comparisons dataset, without loss of performance on the more general SNLI dataset. In this section we consider the performance of the augmented-InferSent model. In this section, we analyze its ability to generalize to 3 different kinds of Xs and Zs that have never been encountered during training.