Categorization in the Wild: Generalizing Cognitive Models to Naturalistic Data across Languages
Table 2: Datasets obtained from language-specific Wikipedia dumps for Arabic (ar), Chinese (ch), English, (en), French (fr), and German (ge). Number of stimuli (concept mentions in linguistic context), concepts, and features (context word types) are shown.
['[EMPTY]', '[BOLD] en', '[BOLD] ar', '[BOLD] ch', '[BOLD] fr', '[BOLD] ge']
From this data set we derived a set of input stimuli as mentions of a concept from the reference set of concepts in sentence context (cf., In order to obtain balanced data sets, we automatically filtered words of low importance to a concept from contexts, using the term-frequency-inverse-document-frequency (tf-idf) metric. After filtering, we only kept stimuli with 3≤n≤20 context words and at most 1,000 stimuli per target concept. The number of stimuli varies across languages as a function of the number of target concepts, and the size of the respective Wikipedia corpus. BCF, nevertheless, achieves purity scores of 0.4 or higher for all languages, meaning that on average at least 40% of the members of a gold standard category are clustered together by BCF (purity rises to 58% for English). This indicates that meaningful categories emerge throughout. The categories shown are intuitively meaningful; in particular vegetable and clothing Cross-lingual comparisons reveal that, compared to BCF, the performance of the co-occurrence model degrades more severely for languages other than English. The number of concepts (i.e., target items to be ranked) differs across languages so that absolute numbers are not directly comparable.