A Unified Tagging Solution:Bidirectional LSTM Recurrent Neural Network with Word Embedding
Table 5: Comparison different word embeddings.
['[BOLD] WE', '[BOLD] Dim', '[BOLD] Vocab#', '[BOLD] Train Corpus (Toks #)', '[BOLD] POS (Acc)', '[BOLD] CHUNK (F1)', '[BOLD] NER']
1 news set. RANDOM is the word embedding set composed of random values which is the baseline. BSLTMWE(10m), BSLTMWE(100m) and BSLTMWE(all) are word embeddings respectively trained by BLSTM-RNN on the first 10 million words, first 100 million words and all 536 million words of North American news corpus. While BSLTMWE(10m) does not bring about obvious improvement, BSLTMWE(100m) and BSLTMWE(all) significantly improve the performance. It shows that BLSTM-RNN can benefit from word embeddings trained by our approach and larger training corpus indeed leads to a better performance. This suggests that the result may be further improved by using even bigger unlabeled dataset. In our experiment, BSLTMWE(all) can be trained in about one day (23 hrs) on a NVIDIA Tesla M2090 GPU. The training time increases linearly with the training corpus size.