Decomposing GeneralizationModels of Generic, Habitual, and Episodic Statements
Table 5: Correlation (ρ) and MAE explained (R1) on test split for argument (top) and predicate (bottom) protocols. Bolded numbers give the best result in the column; the models highlighted in blue are the ones analyzed in §10.
['[EMPTY]', 'Feature sets Type', 'Feature sets Token', 'Feature sets GloVe', 'Feature sets ELMO', '[BOLD] Is.Particular  [ITALIC] ρ', '[BOLD] Is.Particular R1', '[BOLD] Is.Kind  [ITALIC] ρ', '[BOLD] Is.Kind R1', '[BOLD] Is.Abstract  [ITALIC] ρ', '[BOLD] Is.Abstract R1', '[BOLD] All wR1']
We see that (i) our models are generally better able to predict referential properties of arguments than those of predicates; (ii) for both predicates and arguments, contextual learned representations contain most of the relevant information for both arguments and predicates, though the addition of hand-engineered features can give a slight performance boost, particularly for the predicate properties; and (iii) the proportion of absolute error explained is significantly lower than what we might expect from the variance explained implied by the correlations.