Towards Time-Aware Distant Supervisionfor Relation Extraction
Table 4. Noise ratio and set scale of the different training sets.
['[EMPTY]', '[BOLD] Orig. Set', '≥ [BOLD] 0.1', '≥ [BOLD] 0.2', '≥ [BOLD] 0.3', '≥ [BOLD] 0.4', '≥ [BOLD] 0.5', '≥ [BOLD] 0.6']
The Ability to Eliminate the Noises. We investigate the ability of instance-popularity as hard filter to eliminate the noises generated from the DS alignment. Although the instance-popularity has been proved to be useful indirectly a component of Time-DS, a direct evaluation would be much more straightforward. Specifically, we randomly sample 100 training instances respectively from each data sets to check the ratio of noises in each training set. Note that our models which are trained on subset “≥0.3” and subset “≥0.5” achieve the best performance for relation classification and extraction. It is clear that the training sets with the lower ratio of noises tend to be those with which our model achieves the better performance. This is not very strict because the scale of training set also affects the performance.