Towards Instance-Level Parser Selection for Cross-Lingual Transfer of Dependency Parsers
Table 1: Results for single-parser selection models. Results for 42 parsers (an exception is the MSP model which trains a single parser on the concatenation of all training treebanks) on 20 low-resource test languages. Ma & Mi: average performance across 20 languages, macro- and micro-averaged scores, respectively. The best result in each column, not considering oracle scores, is in bold.
['[EMPTY]', 'am', 'be', 'bm', 'br', 'bxr', 'cop', 'fo', 'ga', 'hsb', 'hy', 'kk', 'lt', 'mt', 'myv', 'sme', 'ta', 'te', 'th', 'yo', 'yue', '[BOLD] Ma', '[BOLD] Mi']
Single-parser selection. We report results (UAS) for all single-parser selection methods (i.e., no ensembles) along with the oracle scores on all 20 test treebanks. Allowing for the selection of more than a single parser in cases in which our ILPS-based predictions warrant so (i.e., when two or more parsers yield similarly good performance for some low-resource language) allows SBPSILPS (i.e., its ensemble version, Ens-SBPSILPS) to significantly outperform the strong MSP baseline. However, our treebank-level parser selection model based on instance-level predictions (Ens-SBPSILPS) still significantly outpeforms the ensembles of the other two SBPS methods. As en encouraging finding, both Ens-ILPS and Ens-SBPSILPS outperform the oracle baseline Ens-Or-All, which merges parses produced by all training parsers, using their gold performance on the test treebanks for weighting the individual parser contributions. In summary, we believe these results provide sufficient evidence for the viability of the ILPS transfer paradigm and warrant further research efforts in this direction.