Story Cloze Ending Selection Baselines and Data Examination
Table 3: Experiments using linear classifier with features based on word embeddings. Trained on Dev (tuned with cross-validation) and evaluated on Test.
['[BOLD] Model', '[BOLD] All', '[BOLD] All wo POS sim', '[BOLD] All wo MaxSim', '[BOLD] All wo Sim', '[BOLD] WE S, E1, E2+Sim', '[BOLD] WE E1, E2', '[BOLD] Sims only']
Results on training the feature-based model with The results indicate how well the vector representation models perform in terms of encoding common sense stories. We present the performance of the embedding models depending on the defined features. We perform feature ablation experiments to determine the features which contribute most to the overall score for the different models. 840B. The word2vec model suffers most when similarity features are excluded. We note that the ConceptNet embeddings do not decrease performance when similarity features are excluded, unlike all other models. We also see that the POS similarities are more important than the MaxSim and the Sim (cosine betwen all words in Story and Ending) as they yield worse results, for almost all models, when excluded from All features.