Back to the Future – Sequential Alignment of Text Representations
Table 2: NER (F1 score) on the Broad Twitter Corpus [4].
['[BOLD] Test year', '[BOLD] All', '[BOLD] Same', '[BOLD] Prev', '[BOLD] Unsup.', '[BOLD] Semi-sup.', '[BOLD] Unsup. Unb.', '[BOLD] S. Unb.', '[BOLD] S. Unb. w/Clst']
As with previous experiments, we compare unsupervised and semi-supervised subspace alignment with baselines corresponding to using all data, data from the same year as the evaluation year, and data from the previous year. For each year, we divide the data into 80/10/10 splits for training, development, and test. In the case of NER, we do not observe any positive results for unsupervised subspace alignment. In the case of semi-supervised alignment, however, we find increased performance as compared to training on the previous year, and compared to training on all data. This shows that learning an alignment from just a few data points can help the model to generalise from past data. However, unlike our previous experiments, results are somewhat better when given access to the entire set of training data from the test year itself in the case of NER. The fact that training on only 2013 and evaluating on the same year does not work well can be explained by the fact that the amount of data available for 2013 is only 10% of that for 2012. The identical results for the unbounded extension is because aligning from a single time step renders this irrelevant.