How Can BERT Help Lexical Semantics Tasks?
Table 3: Word analogy prediction accuracy on Google datasets according to different types of word pairs.
['[BOLD] Types', '[BOLD] Example', '[BOLD] SG', '[BOLD] BERT [ITALIC] token', '[BOLD] BERT [ITALIC] avg', '[BOLD] SynGCN', '[BOLD] Ours']
For word analogy, we compare the performances of models according to different types of word pairs. BERTtoken performs relatively lower on “capital-country” and “city-state” compared to skip-gram because it does not model context information. BERTavg improves the results by a large margin, giving comparable results on grammatical related word analogy such as “plural” due to the use of sentential information. SynGCN performs relatively well on grammatical related word pairs by using syntax structures. However, it does not perform well on “capital-country” and “nationality-adjective” word pairs compared with the sequential context based skip-gram model. In contrast, our model takes the advantages of these methods and gives the best overall performance.