Semi-supervised ASR by End-to-end Self-training
Table 2: Dev set performance (measured by %PER) obtained by our method with different W, together with training times, measured as averaged time in seconds spent by each model update, including forwarding and decoding the 32 unsupervised utterances and supervised training on 8+32 utterances. Training time is recorded with a single Tesla K80.
['[EMPTY]', '[ITALIC] W=1', '[ITALIC] W=5', '[ITALIC] W=10', '[ITALIC] W=15']
We now explore the effect of larger W, which intuitively shall give higher pseudo-label quality. For this experiment, we fix other hyperparameters to values found at W=1. Learning curve with W=15 is plotted in Fig. It turns out, with larger W, we can slightly improve the final PER, at the cost of much longer training time (mostly from beam search). Therefore, we recommend using small W with a good base model.