Automatic Quality Estimation for ASR System Combination
Table 14: Performance of different binary classifiers used to find the optimum level of combination.
['[BOLD] Task', '[BOLD] IWSLT', '[BOLD] CHiME-3']
Evaluation metric Balanced accuracy, which is particularly appropriate in case of skewed distributions, is used as evaluation metric to optimize hyper-parameters and select the best classification algorithm. One possible explanation for the success of BayesNet could be in its higher capability to work with a limited number of features (8) and a large number of instances (4496 for IWSLT and 13476 for CHiME-3). Comparing the performance obtained on the two datasets, the fact that CHiME-3 results are better than the IWSLT ones suggests that the more skewed distribution of the IWSLT labels penalizes the classifier.