ner and pos when nothing is capitalized
Table 1: Modern tools trained on cased data perform well on cased test data, but poorly on uncased (lowercased) test data. For NER, we evaluate on the testb set of CoNLL 2003, and the scores are reported as F1. For POS, we evaluate on PTB sections 22-24, and the scores represent accuracy. ELMo refers to contextual representations from Peters et al. (2018).
['Tool', 'Task', 'Test Cased', 'Test Uncased']
But capitalization is not always available. For example, informal user-generated texts can have inconsistent capitalization, and similarly the outputs of speech recognition or machine translation are traditionally without case. Ideally we would like a model to perform equally well on both cased and uncased text, in contrast with current models. Train on uncased Lowercase the training data and retrain. At test time, we lowercase all test data. If we did not do this, then scores on the cased test set would suffer because of casing mismatch between train and test. Since lowercasing costs nothing, we can improve average scores this way. As such, cased and uncased test data will have the same score. Train on cased+uncased Concatenate original cased and lowercased training data and retrain a model. Test data is unmodified. Since this concatenation results in twice the number of training examples than other methods, we also experimented with randomly lowercasing 50% of the sentences in the original training corpus. We refer to this experiment as 3.5 Half Mixed. We also tried ratios of 40% and 60%, but these were slightly worse than 50% in our evaluations. Train on cased, test on truecased Do nothing to the train data, but truecase the test data. Since we lowercase text before truecasing it, the cased and uncased test data will have the same score. Truecase train and test Truecase the train data and retrain. Truecase the test data also. As in experiment 4, cased and uncased test data will have the same score.