SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection
Table 5: Aggregate results on all languages. Bolded results are the ones which beat the best baseline. ∗ and italics denote systems that did not submit outputs in all languages (their accuracy is a partial average).
['System', 'Rank', 'Acc']
This year we had four winning systems (i.e., ones that outperform the best baseline): CULing-01-0, deepspin-02-1, uiuc-01-0, and deepspin-01-1, all neural. Although CULing-01-0 and uiuc-01-0 are both monolingual transformers that do not use any hallucinated data, they follow different strategies to improve performance. The strategy proposed by CULing-01-0 of enriching the input data with extra entries that included non-lemma forms and their tags as a source form, enabled their system to be among top performers on all language families; uiuc-01-0, on the other hand, did not modify the data but rather changed the decoder to be bidirectional and made family-wise fine-tuning of each (monolingual) model. The system is also among the top performers on all language families except Iranian. The third team, DeepSpin, trained and fine-tuned their models on all language data. Both models are ranked high (although the sparsemax model, deepspin-02-1, performs better overall) on most language groups with exception of Algic. Sparsemax was also found useful by CMU-Tartan. The neural ensemble model with data augmentation from IMS team shows superior performance on languages with smaller data sizes (under 10,000 samples). LTI and Flexica teams also observed positive effects of multilingual training and data hallucination on low-resource languages. The latter was also found useful in the ablation study made by NYU-CUBoulder team. Several teams aimed to address particular research questions; we will further summarize their results.