Embedding Learning Through Multilingual Concept Induction
Table 2: Our ten pivot languages, the languages in PBC with the lowest number of types. Tokens in 1000s. Tok Pisin and Bislama are English-based and Sango is a Ngbandi-based creole. PNG = Papua New Guinea
['iso', 'name', 'family; (example) region', 'types', 'tokens']
A simple measure of sparseness that affects embedding learning is the number of types. Fewer types is better since their average frequency will be higher. We randomly sample 5000 verses per edition and compare the number of types based on this selection because most editions do not contain a few of the selected 6458 verses.