A Logic-Driven Framework for Consistency of Neural Models
Table 4: Coverage (%) of unlabeled training sentences during the first epoch of training. Percentages are calculated from models with random seed 1.
['Data', 'M', 'U', 'T']
Specifically, we count the percentage of unlabeled examples where the symmetry/transitivity loss is positive. The coverage decreases in subsequent epochs as the model learns to minimize constraint violations. We see that both datasets M and U have high coverage. The coverage of the dataset T is much lower because the compositional antecedent in transitivity statements holds less often, which naturally leads to smaller coverage, unlike the unary antecedent for symmetry.