NERO: A Neural Rule Grounding Framework forLabel-Efficient Relation Extraction
Table 5. Ablation study of different soft-matching models for Nero on the TACRED dataset.
['Objective', 'Precision', 'Recall', '[ITALIC] F1']
2. Effect of Different Soft-matching Models. Besides word-level attention, we also try other soft matching mechanisms including CBOW-Glove, LSTM+ATT, and BERT-base (both frozen and fine-tuned). All methods first summarize the sentences / rules into vector representations, then calculate the matching scores by cosine similarity. We observe that our method achieves the best performance despite its simplicity. To our surprise, BERT-base performs much worse than our method. In the experiment we find that BERT tends to given high matching scores for all sentence-rule pairs so our framework cannot distinguish the false pseudo-labels from reliable ones.