Augmenting Neural Networks with First-order Logic
Table 2: Impact of constraints on BiDAF. Each score represents the average span F1 on our test set (i.e. official dev set) among 3 random runs. Constrained models and ELMo models are built on top of BiDAF. We set ρ=2 for both R1 and R2 across all percentages.
['%Train', 'BiDAF', '+ [ITALIC] R1', '+ [ITALIC] R2', '+ELMo', '+ELMo, [ITALIC] R1']
Can our framework use rules over named neurons to improve model performance? The answer is yes. We experiment with rules R1 and R2 on incrementally larger training data. We see that our framework can indeed use logic to inform model learning and prediction without any extra trainable parameters needed. The improvement is particularly strong with small training sets. With more data, neural models are less reliant on external information. As a result, the improvement with larger datasets is smaller.