A Comparison of Hybrid and End-to-End Models for Syllable Recognition
Table 2: WER results for experiments with the hybrid system compared to the end-to-end systems
['System', '[BOLD] Hybrid 0-gram', '[BOLD] Hybrid 1-gram', '[BOLD] Hybrid 4-gram', '[BOLD] End-to-End 1 Layer', '[BOLD] End-to-End 3 Layer', '[BOLD] End-to-End 3 Layer + dropout']
The results for experiments performed with kaldi can be found on the left part of Tab. As expected, the experiments with the 0-gram language model show very weak performance since they foremost rely on the AM. This confirms the important role of the LM in hybrid systems. With a WER of 10% the model with the 4-gram LM and the complete lexicon yields the best result overall. There is a huge improvement in WER from the 0-gram to the 1-gram model. Even a very simple LM with no contextual information leads to this huge improvements. With fewer syllables, the WER still improves but also shows problems of LM that rely on learned sequences and lose some of their power when facing lots of unknown words. The test dataset consists of about 37,000 syllable instances of which about 6,800 instances were regarded as OOV (about 18%). This helps to put results for the reduced syllable experiments into perspective since decoding performance suffers significantly from a high OOV rate. Tab. The results indicate that more layers lead to better performance, with a best WER result of 27.53% using the reduced number of syllables, dropout for regularization and three hidden layers. This confirms the assumption that deeper networks perform better and the network does better with fewer training targets. The difference in performance between reduced and full syllable sets almost vanishes for the networks where no dropout was performed. However, the reduction of targets does not appear to be a deciding factor: Relative improvement is only about 7.5% in WER and does not appear reasonable compared to the amount of information lost. On the other hand, adding dropout and more layers leads to a 24% relative improvement in performance for the experiments with the complete target set.