Uncovering Probabilistic Implications in Typological Knowledge Bases
Table 1: Accuracies for feature prediction in a typologically diverse test set, across number of implicants used. Note that the numbers are not comparable across columns nor to the baseline, since each makes a different number of predictions.
['N implicants', '2', '3', '4', '5', '6']
We split our evaluation of our model up across the feature categories present in WALS. During the typological feature prediction experiments, we consider a single such WALS category at a time. We vary the number of implicants by allowing the model to observe 2 to 6 features from within this category as well as the values of features in other categories. This is done as having access to, e.g., all word-order features when predicting a final word order feature would be much easier than our setting. Hence, our experiment will show the extent to which increasing the number of features from the current feature category affects predictive power. We vary the number of implicants k from 2 to 6 features in each category with a total of n features, this gives us (nk) total sets per number of implicants k. For each such set, we attempt to predict all held-out features in that category in a leave-one-out-style evaluation. This results in (nk)(n−k) predictions to make per category per number of implicants k. Performance is measured by averaging the accuracy of predictions of all held-out features over the entire test set, across categories.