Cross-Domain Generalization of Neural Constituency Parsers
Table 5: F1 and exact match accuracies comparing the Chart (unstructured) and In-Order (structured) parsers with BERT pretraining on English (top) and Chinese (bottom) corpora.
['[EMPTY]', 'F1 Chart', 'F1 In-Order', 'Exact Match Chart', 'Exact Match In-Order']
However, these aggregate F1 scores decompose along the structure of the tree, and are dominated by the short spans which make up the bulk of any treebank. Structured-conditional prediction may plausibly be most useful for predicting larger portions of the tree, measurable in exact match accuracies and in F1 on longer-length spans (containing more substructure). First, we compare the tree-level exact match accuracies of the two parsers. In fact, for several corpora (Blogs and B. Conv) the In-Order parser outperforms the Chart parser on exact match despite having the same or lower F1. This suggests that conditioning on structure in the model induces a correlation between span-level decisions that becomes most apparent when using a metric defined on the entire structure.