How to Generate a Good Word Embedding?
Table 10: The number of cases in which the strategy wins if iteration is stopped when the specified task peaks.
['[BOLD] loss', '[BOLD] syn', '[BOLD] sem', '[BOLD] ws', '[BOLD] tfl', '[BOLD] avg', '[BOLD] ner', '[BOLD] cnn', '[BOLD] pos']
Despite the inconsistency between the validation loss and the task performance, it is worth noting that most tasks always exhibit similar peaks. The results demonstrate that we can use a simple task to verify whether the word embedding has peaked on other tasks. We consider the various combinations of the eight tasks, the seven models, and the three subsets of the W&N corpus, thereby obtaining 168 tuples. If we use the strategy that stop at the peak of the loss on the validation set, the resulting trained model will win in 89 cases. If we use the strategy that stops at the peak of the tfl task (the simplest task in our experiments), it will win in 117 cases.