Augmenting Neural Networks with First-order Logic
Table 4: Impact of constraints on BiLSTM tagger. Each score represents the average accuracy on test set of 3 random runs. The columns of +CRF, +C1:5, and +CRF,C1:5 are on top of the BiLSTM baseline. For C1:4, ρ=4 for all percentages. For C5, ρ=16.
['%Train', 'BiLSTM', '+CRF', '+ [ITALIC] C1:5', '+CRF, [ITALIC] C1:5']
S4SS3SSS0Px3 How does local augmentation compare with global inference? While a first-order Markov model (\eg, the BiLSTM-CRF) can learn pairwise constraints such as C1:4, we see that our framework can better inform the model. Interestingly, the CRF model performed even worse than the baseline with ≤40% data. This suggests that global inference relies on more training examples to learn its scoring function. In contrast, our constrained models performed strongly even with small training sets. And by combining these two orthogonal methods, our locally augmented CRF performed the best with full data.