Back to the Future – Sequential Alignment of Text Representations
Table 1: Paper acceptance prediction (acc.) on the PeerRead data set [10]. Abbreviations represent Unsupervised, Semi-supervised, Unsupervised Unbounded, Semi-supervised Unbounded, and Semi-supervised Unbounded with Clustering.
['[BOLD] Test year', '[BOLD] All', '[BOLD] Same', '[BOLD] Prev', '[BOLD] Unsup.', '[BOLD] Semi-sup.', '[BOLD] Unsup. Unb.', '[BOLD] S. Unb.', '[BOLD] S. Unb. w/Clst']
With unsupervised subspace alignment, we observe relatively unstable results – in one exceptional case, namely testing on 2010, unsupervised alignment is as helpful as semi-supervised alignment. Semi-supervised alignment, however, yields consistent improvements in performance across the board. It is especially promising that adapting from past data outperforms training on all available data, as well as training on the actual in-domain data. This highlights the importance of controlling for data drift due to language evolution. It shows that this signal can be taken advantage of to increase performance on present data with only a small amount of annotated data. We further find that using several past time steps in the Unbounded condition is generally helpful, as is using instance similarities in the alignment.