Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning
Table 3: Accuracy of various baselines (italics), our system (DQN) and the Oracle on Shootings and Adulteration datasets. Agg. refers to aggregation baselines. Bold indicates best system scores. ∗statistical significance of p<0.0005 vs basic Maxent extractor using the Student-t test. Numbers in parentheses indicate the optimal threshold (τ) for the aggregation baselines. Confidence-based reconciliation was used for RL-Query.
['[BOLD] System', '[BOLD] Shootings  [BOLD] ShooterName', '[BOLD] Shootings  [BOLD] NumKilled', '[BOLD] Shootings  [BOLD] NumWounded', '[BOLD] Shootings  [BOLD] City', '[BOLD] Adulteration  [BOLD] Food', '[BOLD] Adulteration  [BOLD] Adulterant', '[BOLD] Adulteration  [BOLD] Location']
For instance, RL-Extract is 11.4% more accurate than the basic Maxent extractor on City and 7.1% better on NumKilled, while also achieving gains of more than 5% on the other entities on the Shootings domain. The gains on the Adulteration dataset are also significant, up to a 11.5% increase on the Location entity.