Sentence Embedding Alignment for Lifelong Relation Extraction
Table 1: The average accuracy across all the tasks at last time step for EMR and GEM on both non-NLP and our lifelong relation detection benchmarks. For the experiments on MNIST and CIFAR, we follow the setting in Lopez-Paz and Ranzato (2017) (see Appendix A.2 for details). For the experiments on FewRel and SimpleQuestions, we use the same setting in Section 6. We only implement task-level EMR for MNIST and CIFAR because of the relatively easy implementation.
['Task', 'EMR sample', 'EMR task', 'GEM']
However, the sample-level strategy could sometimes make the code implementation more difficult: for some lifelong learning benchmarks such as MNIST Rotation, MNIST Permutation, and CIFAR-100 used in Lopez-Paz and Ranzato The numbers are the average accuracy, i.e. 1k∑kj=1accf,j, at last time step. For both algorithms, the training data is randomly sampled to store in the memory, following Lopez-Paz and Ranzato On lifelong relation detection, the EMR outperforms GEM on both of our created benchmarks. To further show its generalizability, we apply the EMR to previous lifelong MNIST and CIFAR benchmarks and compare to the results in Lopez-Paz and Ranzato Still, EMR performs similarly to GEM except for the MNIST Rotation benchmark.