Approximation-Aware Dependency Parsing by Belief Propagation
Table 2: Results on 19 languages from CoNLL-2006/2007. For languages appearing in both datasets, the 2006 version was used, except for Chinese (zh). Evaluation follows the 2006 conventions and excludes punctuation. We report absolute UAS for the baseline (CLL) and the improvement in UAS for L2 over CLL (L2-CLL) with positive/negative differences in blue/red. The average UAS and average difference across all languages (Avg.) is given.
['[EMPTY]', '1st-order', '1st-order', '2nd-order (With given num. BP iterations) 1', '2nd-order (With given num. BP iterations) 1', '2nd-order (With given num. BP iterations) 2', '2nd-order (With given num. BP iterations) 2', '2nd-order (With given num. BP iterations) 4', '2nd-order (With given num. BP iterations) 4', '2nd-order (With given num. BP iterations) 8', '2nd-order (With given num. BP iterations) 8']
S5SS2SSS0Px4 Other Languages We find that, on average, approximation-aware training with an L2 objective obtains higher UAS than CLL training. This result holds for both 1st- and 2nd-order models with grandparent and sibling factors with 1, 2, 4, or 8 iterations of BP. We find that the approximation-aware training doesn’t always outperform CLL training—only in the aggregate. Again, we see the trend that our training approach yields more significant gains when BP is restricted to a small number of maximum iterations.