Improving distant supervision using inference learning
Table 3: Evaluation using manually labelled data
['relation', '[BOLD] Unfiltered Prec.', '[BOLD] Unfiltered Rec.', '[BOLD] Unfiltered F1', '[BOLD] Random-reduced Prec.', '[BOLD] Random-reduced Rec.', '[BOLD] Random-reduced F1', '[BOLD] PRA-reduced Prec.', '[BOLD] PRA-reduced Rec.', '[BOLD] PRA-reduced F1']
The best overall performance is once again obtained using the PRA-reduced classifier. There is an increase in recall for both relations and a slight increase in precision for may_treat. Performance of the Random-reduced classifier also improves due to an increasing recall but remains below PRA-reduced. Performance of the Random-reduced classifier is also better than Unfiltered, with the overall improvement largely resulting from increased recall, but below PRA-reduced. These results confirm that removing examples identified by PRA improves the quality of training data.