Hierarchical Neural Networks for Sequential Sentence Classification in Medical Scientific Abstracts
Table 9: Comparison of performance with different choices of word embeddings for our HSLN-RNN model trained on the PubMed 20k dataset (reported on F1-scores on the test set). “P.M.” means PubMed.
['2.5 Embedding', 'Dimension', 'P.M. 20k']
In order to test the importance of pretrained word embeddings, we performed experiments with different sets of publicly published word embeddings, as well as our locally curated word embeddings, to initialize our model. The combination of Wikipedia and PubMed abstracts as the corpus for unsupervised word embedding training yields the best result, and the individual use of either the Wikipedia corpus or the PubMed abstracts performs much worse.