Syntax-Aware Multi-Sense Word Embeddingsfor Deep Compositional Models of Meaning
Table 5: Results with average pooling for the paraphrase detection task (accuracy × 100).
['[BOLD] Model', '[BOLD] Ambig.', '[BOLD] Prior', '[BOLD] Dynamic']
In the above case, the contribution of the words within the verb phrase to the final sentence representation will be faded out due to the recursive composition mechanism. Inspired by related work in computer vision By doing this we expect that the new sentence vector will reflect local features from all words in the sentence that can help in the classification in a more direct way. More importantly, the effect of dynamic disambiguation now becomes more significant, as expected by our analysis. We see that, despite the large size of the training set, the results are much lower than the ones obtained when using the pre-training step. This demonstrates the importance of the initial training on a general-purpose corpus: the resulting vectors reflect linguistic information that, although not obtainable from the task-specific training, can make great difference in the result of the classification.