Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition
Table 2: The hyper-parameters used in the experiments
['Hyper-parameter', 'Value']
For an unbiased evaluation, all the trained models have been tested blindly on unseen test data. In order to facilitate replication of the empirical results, we have used a publicly-available library for the implementation of the neural networks (i.e. the Theano neural network toolkit bergstra2010theano) and we release our code. Therefore, we have divided the training set of each dataset into two parts: a training set for learning the parameters (70%), and a validation set (30%) for selecting the best hyperparameters bergstra2012random. The hyper-parameters of the LSTM include the number of hidden nodes (for both LSTM versions), (Hw,Hc)∈{25,50,100}; the word embedding dimension, dw∈{50,100,200,300,600}; and the character embedding dimension, dc∈{25,50,100}. Additional hyper-parameters include the learning rate and the drop-out rate, which were left to their default values of [0.01] and [0.5] respectively srivastava2014dropout. All weights in the network, feature encodings and the words that do not have a pre-trained word embedding have been initialized randomly from the uniform distribution within range [−1, 1], and updated during training with backpropagation. The number of training ”epochs” (i.e., iterations) was set to 100, selecting the epoch that obtained the best results on the validation set. The best model from the validation set was finally tested on the unseen, independent test set without any further tuning, and the corresponding accuracy reported in the tables.