Automatic Quality Estimation for ASR System Combination
Table 10: WER[%] (↓) of different ranking methods on the CHiME-3 training set (DT05) using 8-fold CV.
['[BOLD] Ranking', 'L3', 'L4', 'L5', 'L6', 'L7', 'L8', 'L9', 'L10', 'L11', 'L12', 'L13', 'L14', 'Avg.']
Unlike IWSLT, most of our results outperform the SysO oracle, which always uses the enhanced channels as the first input. The best WERs are usually obtained at the lower levels (i.e. 9.5% at L4, L5 and L6). This can be explained by the large gap between the best channels (ds−dnn and ds−gmm, respectively obtaining 10.4% and 12.2% WER) and the other channels (obtaining from 16.5% to 24.2% WER). By adding more hypotheses, worse transcriptions are considered in the combination and, consequently, majority voting makes more mistakes. At L13 and L14, indeed, our results remain slightly worse than SysO.