How to Generate a Good Word Embedding?
Table 4: Best results for 50-dimensional embeddings train by each model on the W&N corpus (2.8B tokens).
['[BOLD] Model', '[BOLD] syn', '[BOLD] sem', '[BOLD] ws', '[BOLD] tfl', '[BOLD] avg', '[BOLD] ner', '[BOLD] cnn', '[BOLD] pos']
First, the performance metrics for the different tasks lie in different ranges. For each model, we iterate until the model converges to or overfits all of the tasks. Overall, the best results are obtained using various models, from GloVe to LBL. Compared with random embedding, all investigated word embeddings demonstrate better performance on the tasks regardless of the model used. Second, for tasks in which the word embedding is used as a feature or for the initialization of neural networks, the choice of model does not significantly affect the results. Therefore, simpler models are typically sufficient for real tasks.