USFD at KBP 2011: Entity Linking, Slot Filling and Temporal Bounding
Table 8: % Slot Values Correctly Extracted
['Dataset', 'DATE', 'PER', 'LOC', 'ORG']
In addition to evaluating the quality of the training data we generated, we did some evaluation to determine the optimal feature set combination. Finally we evaluated the slot value extraction capabilities on a small test set of example sentences selected from the source collection to ensure they contained the target entity and the correct answer, as well as some negative instances, and manually processed to correctly annotate the entities within them (simulating perfect upstream performance). The large difference in performance between the ten fold cross validation figures over the training and the evaluation against the small handpicked and annotated gold standard from the source collection may be due to the fact that the training data was Wikipedia texts while the test set is news texts and potentially other text types such as blogs; however, the handpicked test set is very small (70 sentences total) so generalizations may not be warranted.