Use Generalized Representations, But Do Not Forget Surface Features
Table 2: Results on the WikiCoref dataset.
['[EMPTY]', 'Non-Anaphoric R', 'Non-Anaphoric P', 'Non-Anaphoric F1', 'Anaphoric R', 'Anaphoric P', 'Anaphoric F1']
The WikiCoref dataset is annotated according to the same annotation guideline as that of CoNLL. Therefore, it is an appropriate dataset for performing out-of-domain evaluations when CoNLL is used for training. The word dictionary that is used for the LSTM model is built based on the CoNLL 2012 training data. All words that are not included in this dictionary are treated as out of vocabulary words with randomly initialized word embeddings. We further improve the performance of LSTM on WikiCoref, by adding the words from the WikiCoref dataset into its dictionary. LSTM† results are still lower than those of the SVM model while SVM does not use any information from the test dataset. Pruning rare lexical features from the training data along the incorporation of part of speech tags, which are far more generalizable than lexical features, could explain the generalizability of the SVM model on the new domain.