Drug-Drug Interaction Extraction from Biomedical Text Using Long Short Term Memory Network
Table 3: Values of different regularization parameters used in the three models.
['[ITALIC]  [BOLD] Models', '[ITALIC]  [BOLD] Dropout', '[ITALIC] l2 [ITALIC]  [BOLD]  regu.']
As there is no separate development or validation set available, we divided the original training dataset into two parts, 80% as training and rest 20% as validation sets. Hyper-parameters are tuned using this validation set. Hidden layer size in B-LSTM and AB-LSTM are kept as 200, and 150 for Joint AB-LSTM. Pre-trained word embedding of 100 dimensions and distance embedding of 10 dimensions are used in all three LSTM models. We used both l2 regularization and dropout We applied dropout only on the output of the pooling layers. Next to decide about the early-stopping criteria (i.e. number of epochs neural network models will be trained), we again split the original training dataset into two parts containing 95% and 5% of all samples. Now we trained the model on 95% set using above selected hyper-parameters and selected the epoch giving the best result on the left out 5% of the original training data. Trained model on the 95% set is used on the independent test set and corresponding results are discussed in the respective sections.