Bridging linguistic typology and multilingual machine translation with multi-view language representations
Table 5: BLEU score average per language family (IE=Indo-European). Every method includes the weighted BLEU average per number of languages (#L) and the number of clusters/models. Bold and italic represent first and second best results per family. for SVCCA indicates the difference with respect to the highest score.
['Lang. families Isolate (Basque)', '# L 1', 'Size (k) 5', 'Individual 2.20', 'Massive  [BOLD] 11.10', 'Family 2.20', '[ITALIC] US  [ITALIC] 10.90', '[ITALIC] LT 5.60', '⊕ 3.90', '[BOLD] SVCCA-53 6.40', '[BOLD] SVCCA-53 -4.7', '[BOLD] SVCCA-23 10.10', '[BOLD] SVCCA-23 -1.0']
Following a guide for evaluating multilingual benchmarks anastasopoulos-2019-multilingual, we also group the scores by language families. We observe that most of the approaches have obtained clusters with similar overall translation accuracy. The individual models are the only ones that significantly underperform. The poor performance is transferred to the Family baseline, as most of the groups contains only one language given the low language diversity of the dataset.