End-to-end Anchored Speech Recognition
Table 2: Augmented vs. Device-directed-only training data. Results with “Device-directed-only” training set are from Table 1 for clearer comparisons.
['Model', 'Training Set', 'Test Set', 'WER', 'sub', 'ins', 'del', 'WERR(%)']
Next, we further validate the effectiveness of the Multi-source Attention model by showing how synthetic training data has different impact on it and the baseline model respectively. Synthetic training data is prepared such that 50% of the utterances in the training set are kept unchanged, 44% are processed with Synthetic Method 1, and 6% are processed with Synthetic Method 2. The ratio is tuned on the development set. This new training data is referred as “augmented” in all result tables. For the baseline model, the performance degrades drastically when trained on augmented data: the deletion errors on both of the “normal” and “hard” test sets get much higher. This is expected since without the anchor word the model has no extra acoustic information of which part of the utterance is desired, so that it tends to ignore frames regardless of whether they are actually from device-directed speech. On the contrary, for the Multi-source Attention model the WERR (augmented vs. device-directly-only) on the “hard” set is 12.5%, and WER on the “normal” set does not get worse. Moreover, the insertion errors on both test sets get reduced while the deletion errors increase much less than those in the case of the baseline model, indicating that by incorporating the anchor word information the proposed model effectively improves the ability of focusing on device-directed speech and ignoring others. This series of experiments also reveals significant benefits from using the synthetic data with the proposed model. In total, the combination of the Multi-source Attention model and augmented training data achieves 14.9% WERR on the “hard” set, with only 1.5% degradation on the “normal” set.