Cross-Domain Generalization of Neural Constituency Parsers
Table 1: Performance and relative increase in error (both given by F1) on English corpora as parsers are evaluated out-of-domain, relative to performance on the in-domain WSJ Test set. Improved performance on WSJ Test translates to improved performance out-of-domain. The two parsers with similar absolute performance on WSJ (BLLIP and In-Order) have comparable generalization out-of-domain, despite one being neural and one non-neural.
['[EMPTY]', 'Berkeley F1', 'Berkeley Δ Err.', 'BLLIP F1', 'BLLIP Δ Err.', 'In-Order F1', 'In-Order Δ Err.', 'Chart F1', 'Chart Δ Err.']
(Chart, In-Order). None of these parsers use additional data beyond the WSJ training section of the PTB: we use the version of the BLLIP parser without self-training on unlabeled data, and use the In-Order parser without external pre-trained word embeddings. Across all parsers, higher performance on the WSJ Test set corresponds to higher performance on each out-of-domain corpus, showing that the findings of McClosky et al. In particular, the Chart parser has highest performance in all four domains.