Nested Named Entity Recognition via Second-best Sequence Learning and Decoding
Table 2: Hyperparameters in our experiments.
['[BOLD] Hyperparameter', '[BOLD] Value']
In this study, we adopt as baseline a BiLSTM-CRF model, which is widely used for NER tasks Lample et al. We apply our usage of CRF to this baseline. We prepare three types of models for fair comparisons with existing methods. The first one is the model to which is fed conventional word embeddings and CNN-based character-level representation Ma and Hovy We initialize word embeddings with the pretrained embeddings GloVe Pennington et al. For GENIA, we adopt the pretrained embeddings trained on MEDLINE abstracts Chiu et al. The initialized word embeddings are fixed during training. The vectors of the word embeddings and the character-level representation are concatenated and then input into the BiLSTM layer. The second model is the model combined with the pretrained BERT model Devlin et al. We use the uncased version of BERT large model as a contextual word embeddings generator without fine-tuning and stack the BiLSTM layers on top of the BERT model. The third model is the BiLSTM-CRF model to which is fed word embeddings, character-level representation, BERT embeddings, and FLAIR embeddings Akbik et al. using FLAIR framework Akbik et al. All our models have 2 BiLSTM hidden layers, and the dimensionality of each hidden unit is 256 in all our experiments. We adopt AdaBound Luo et al. Early stopping is used based on the performance of development set. We repeat the experiment 5 times with different random seeds and report average and standard deviation of F1-scores on a test set as the final performance.