M2H-GAN: A GAN-based Mapping from Machine to Human Transcripts for Speech Understanding
Table 2: Accuracies obtained by various models on the DECODA corpus. “Real Test” stands for the performances observed on the test set w.r.t to the validation set, while “Max Test” are the best results obtained. Results are averaged over 10 runs. The standard deviation is computed over these runs and concern the “Real Test” performances.
['[BOLD] Models', '[BOLD] Data', '[BOLD] Dev.', '[BOLD] Real Test', '[BOLD] Max Test', '[BOLD] Std. Dev.']
It is first important to note the difference in term of accuracy, between the two baselines during the theme identification on both ASR and TRS transcripts. Indeed, while the standard deviation remains almost equal, both real (w.r.t to the validation set) and max test accuracies are different. More precisely, the ASR-based DNN obtains a real test accuracy of 83.4% compared to 88.0% for the TRS-based DNN, representing a drop of 4.6%. This is easily explained by the high WER observed on the ASR transcriptions These results support the initial intuition that a translation of ASR documents to TRS-like representations allow us to better identify the most related theme of a spoken dialogue.