“Translation can’t change a name”: Using Multilingual Data for Named Entity Recognition
Table 4: F1 scores of NER systems trained using noisy twitter data.
['NER', 'Baseline', 'Twitter clusters']
The labeled NE training data for all of the languages we have used comes from the newswire domain. Thus the news-commentary data (cf. Sec. Since we cannot always expect to obtain in-domain data, we use word clusters 850 million tokens clustered into 1000 classes for generalization For English, German and French we again obtain a highly significant improvement; however the improvements obtained using the out-of-domain data are lesser than that obtained using in-domain data.