A Comparison of Word Embeddings for English and Cross-Lingual Chinese Word Sense Disambiguation
Table 4: Accuracy of a basic LSTM approach on the Lexical Sample and All Words tasks.
['[BOLD] Method', '[BOLD] SE-2 LS', '[BOLD] SE-3 LS', '[BOLD] SE-2 AW', '[BOLD] SE-3 AW']
The models converge to just using the most common sense for the AW task. A possible reason for this is overfitting. WSD is known to suffer from data sparsity. Although there are many training examples in total, as we train a separate model for each word, many individual words only have few training examples. We note other attempts to use neural networks for WSD may have run into the same problem. Taghipour and Ng \shortciteTaghipour15 indicated the need to prevent overfitting while using a neural network to adapt C&W embeddings by omitting a hidden layer and adding a Dropout layer, while Kågebäck and Salomonsson \shortcitekaageback2016word developed a new regularization technique in their work.