Augmenting Neural Networks with First-order Logic
Table 3: Impact of constraints on L-DAtt network. Each score represents the average accuracy on SNLI test set among 3 random runs. For both N1 and N2, we set ρ=(8,8,8,8,4) for the five different percentages. For the noisy constraint N3, ρ=(2,2,1,1,1).
['%Train', 'L-DAtt', '+ [ITALIC] N1', '+ [ITALIC] N2', '+ [ITALIC] N3', '+ [ITALIC] N2,3']
S4SS2SSS0Px3 How does our framework perform with large training data? The SNLI dataset is a large dataset with over half-million examples. The straightforward constraint N1 performs strongly with ≤2% data while its conservative alternative N2 works better with a larger set. However, with full dataset, our augmented models perform only on par with baseline even with lowered scaling factor ρ. These observations suggest that if a large dataset is available, it may be better to believe the data, but with smaller datasets, constraints can provide useful inductive bias for the models.