Choosing the Right Word: Using Bidirectional LSTM Tagger for Writing Support Systems
Table 3: Lower bound (underestimate) MRR for the scientific test set. The proposed bidirectional LSTM model trained on a domain specific corpus substantially outperforms multiple baselines which also employ only unlabeled data for constructing a statistical classifier.
['[BOLD] Model', '[BOLD] MRR']
Word embedding dimension = 200 Number of LSTM hidden units = 200 Sentential context embedding dimension = 400 Training batch size = 100 sentences Maximal sentence length = Vocabulary size = 30,000 (the size of initial one-hot vectors, representing the words as the starting point for word embedding learning) N-gram LM with Kneser-Ney smoothing was trained to learn a distribution for five n-gram’s sizes, starting from unigram to five-gram.