Data Augmentation for Training Dialog Models Robust to Speech Recognition Errors
Table 2: Accuracy (%) and F1-score results for reduced model architectures. Except for model architecture, the settings are exactly the same as those of Table 1. “without BERT” refers to the model architecture from Table 1 with BERT replaced with GloVe embeddings; “Simple NN” refers to the GloVe embeddings + 1 layer unidirectional LSTM with 128 hidden size architecture.
['[BOLD] Training  [BOLD] Setup', '[BOLD] Model  [BOLD] Architecture', '[BOLD] Hypothesis  [BOLD] Acc.', '[BOLD] Hypothesis  [BOLD] F1', '[BOLD] Reference  [BOLD] Acc.', '[BOLD] Reference  [BOLD] F1']
To investigate whether data augmentation is more effective for models without large pre-trained embeddings like BERT, we run one set of follow-up experiments where we replace BERT with GloVe embeddings Pennington et al. GloVe embeddings + 1 layer unidirectional LSTM with 128 hidden size (a further 83.05% reduction in number of parameters). We see that for the reduced model architectures, the improvements from data augmentation are much larger (e.g., 3.89% absolute improvement in S1 and 2.34% in S2 for Simple NN). The promising results on simpler model architectures have practical implications, because in certain use cases we may not be able to use large model architectures due to constraints on model size, computing power, or latency.