Mapping Unseen Words to Task-Trained Embedding Spaces
Table 1: Results of dependency parsing on various treebanks. Entries of the form A→B give results for parsing without mapped embeddings (A) and with mapped embeddings (B). “OOTV %” entries A→B indicate that A% of the test set vocabulary was unseen in the parser training, and B% remain unknown after mapping the embeddings. “OOTV UAS” refers to UAS measured on the subset of the test set sentences that contain at least one OOTV word, and “#Sents” gives the number of sentences in this subset.
['[EMPTY]', 'Lower OOTV word rate WSJ', 'Lower OOTV word rate OntoNotes', 'Lower OOTV word rate Avg.', 'Higher OOTV word rate Answers', 'Higher OOTV word rate Emails', 'Higher OOTV word rate Newsgroups', 'Higher OOTV word rate Reviews', 'Higher OOTV word rate Weblogs', 'Higher OOTV word rate Avg.']
Due to the small initial OOTV rates (<3%), we only see modest gains of 0.3-0.4% in UAS, with statistical significance at p<0.05 for WSJ and OntoNotes and p<0.07 for Switchboard. The initial OOTV rates are cut in half by our mapper, with the remaining unknown words largely being numerical strings and misspellings. When only considering test sentences containing OOTV words (the row labeled “OOTV subset”), the gains are significantly larger (0.5-0.8% UAS at p<0.05). As expected, the mapper has a much larger impact when parsing these out-of-domain datasets with high OOTV word rates.