KAS-term: Extracting Slovene Terms from Doctoral Theses via Supervised Machine Learning
Table 2: Area under curve (AUC) results when exploiting single features and the combination of all features (all). We discriminate between the three areas and the multi-word (MWT) and single-word terms (SWT).
['[EMPTY]', 'Chemistry', 'Computer', 'Political', 'Overall']
We first evaluate the term predictability of each of the explanatory variables in isolation by performing ranking experiments. As our response variable we use the rank of the term given the chosen statistic. Next we run an SVM regressor with an RBF kernel on all available variables, using the average annotation of each candidate (we have four annotations per candidate) as our response variable. For obtaining our predictions, we use cross-validation, in each iteration leaving one out of 15 available theses out for testing / annotation, and training on the remaining 14 doctoral theses. We evaluate each ranking by calculating the area under the curve (AUC) score which is a very convenient estimate as it does not require any decision on a threshold, i.e., the precision and recall trade-off.