AidMe
Table 2. Comparison on the SemEval dataset between AidMe, baseline models and the best teams of the competition
['[BOLD] Model', '[BOLD] Description', '[BOLD] Pearson corr.']
, we have evaluated its performance on the STS Benchmark We evaluate on the test set a model that was trained on the train set. We report the results of three of the best teams and two baselines computed by the organizers: cosine similarity on sentence embedding obtained with InferSent (Conneau2017) or by averaging the word embeddings of a pre-trained word embedding (Paragram (Wieting2015)). We do not intend to compete with the best models but to ensure that our model is robust enough, stays close to the state of the art models while being computationally efficient. We see that our model is well above simple baselines and not too far from state of the art performance even though it could not beat InferSent which is very strong on this dataset. We show below that InferSent performs much worse on the intent detection task and we could not use it for AidMe.