Parameter sharing between dependency parsers for related languages
Table 4: Performance on development data (LAS; in %) across select sharing strategies for unrelated languages. Mono is our single-task baseline; Language-best is using the best sharing strategy for each language (as evaluated on development data); Best and Worst are the overall best and worst sharing strategy across languages; Char shares only the character-based LSTM parameters; Word shares only the word-based LSTM parameters; All shares all parameters. refers to hard sharing, ID refers to soft sharing, using an embedding of the language ID and ✗ refers to not sharing.
['[BOLD] Model', '[BOLD] C', '[BOLD] W', '[BOLD] S', 'he', 'no', 'fi', 'hr', 'ru', 'es', 'it', 'et', 'nl', 'ar', 'Av']
We repeated the same set of experiments with unrelated language pairs. We hypothesise that parameter sharing between unrelated language pairs will be less useful in general than with related language pairs. However, it can still be useful, it has been shown previously that unrelated languages can benefit from being trained jointly. For example, Lynn et al. The table only presents part of the results, the rest can be found in the supplementary material. As expected, there is much less to be gained from sharing parameters between unrelated pairs. However, it is possible to improve the monolingual baseline by sharing some of the parameters. In general, sharing the MLP is still a helpful thing to do. It is most helpful to share the MLP and optionally one of the two other sets of parameters. Results are close to the monolingual baseline when everything is shared. Sharing word and character parameters but not the MLP hurts accuracy compared to the monolingual baseline.