Incorporating Syntactic and Semantic Information in Word Embeddings using Graph Convolutional Networks
Table 5: Comparison of ELMo with SynGCN and SemGCN embeddings on multiple extrinsic tasks. For each task, models use a linear combination of the provided embeddings whose weights are learned. Results show that our proposed methods encode complementary information which is not captured by ELMo. Please refer Section 9.4 for more details.
['Method', 'POS', 'SQuAD', 'NER', 'Coref']
Recently, ELMo Peters et al. In this section, we evaluate SynGCN and SemGCN when given along with pre-trained ELMo embeddings. The results show that dependency context based embeddings encode complementary information which is not captured by ELMo as it only relies on sequential context. Hence, our proposed methods serves as an effective combination with ELMo.