Weakly-supervised Relation Extraction by Pattern-enhanced Embedding Learning
Table 3. Quantitative results on the RE task.
['[BOLD] Algorithm', '[BOLD] Wiki + Freebase  [BOLD] P@50', '[BOLD] Wiki + Freebase  [BOLD] R@50', '[BOLD] Wiki + Freebase  [BOLD] F1@50', '[BOLD] Wiki + Freebase  [BOLD] P@100', '[BOLD] Wiki + Freebase  [BOLD] R@100', '[BOLD] Wiki + Freebase  [BOLD] F1@100', '[BOLD] NYT + Freebase  [BOLD] P@50', '[BOLD] NYT + Freebase  [BOLD] R@50', '[BOLD] NYT + Freebase  [BOLD] F1@50', '[BOLD] NYT + Freebase  [BOLD] P@100', '[BOLD] NYT + Freebase  [BOLD] R@100', '[BOLD] NYT + Freebase  [BOLD] F1@100']
2. Corpus-level Relation Extraction (RE). Next, we show the results on the corpus-level relation extraction task. For the approaches using textual patterns (PATTY, Snowball), we see the results are quite limited especially on the NYT dataset. For other neural network based pattern approaches (PathCNN, CNN-ATT, PCNN-ATT), although they are proved to be very effective when the given instances are abundant, their performance in the weakly-supervised setting is not satisfactory. The reason is that they typically deploy complicated convolutional layers or recurrent layers in their model, which rely on massive relation instances to tune. However, in our setting, the instances are very limited, leading to their poor performance. For the integration approach (LexNET), although it incorporates the distributional information, the performance is still quite limited especially on the NYT dataset. This is because the joint training framework of LexNET also requires considerable training instances.