Improved Relation Extraction withFeature-Rich Compositional Embedding Models
Table 3: Comparison of models on ACE 2005 out-of-domain test sets. Baseline + HeadOnly is our reimplementation of the features of nguyen_employing_2014.
['[EMPTY]', '[BOLD] Model', '[BOLD] P', '[BOLD] R', '[BOLD] cts  [BOLD] F1', '[BOLD] cts  [BOLD] P', '[BOLD] cts  [BOLD] R', '[BOLD] wl  [BOLD] F1', '[BOLD] wl  [BOLD] P', '[BOLD] wl  [BOLD] R', '[BOLD] Avg.  [BOLD] F1', '[BOLD] F1']
In the typical gold entity spans and types setting, both \newciteplank_embedding_2013 and \newcitenguyen_employing_2014 found that they were unable to obtain improvements by adding embeddings to baseline feature sets. By contrast, we find that on all domains the combination baseline + fcm (5) obtains the highest F1 and significantly outperforms the other baselines, yielding the best reported results for this task. We found that fine-tuning of embeddings (2) did not yield improvements on our out-of-domain development set, in contrast to our results below for SemEval. We suspect this is because fine-tuning allows the model to overfit the training domain, which then hurts performance on the unseen ACE test domains.