End-To-End Visual Speech Recognition With LSTMs
Table 2: Classification Accuracy on the CUAVE database. The end-to-end model is trained using the same protocol as [4, 23] where 18 subjects are used for training and validation and 18 for testing. ∗ This model is trained on 28 subjects and tested on 8 subjects. † These models are trained and tested using a 6-fold cross validation. ‡ This model uses a visual front-end which is significantly more complicated than ours.
['Method', 'Classification']
There is not a standard evaluation protocol for this database which makes comparison between different works difficult. We see that the single-stream end-to-end model based on raw mouth ROIs outperforms both previous works. The 2-stream end-to-end model outperforms all approaches that use a similar visual front-end. In this study, we use much fewer subjects, 18, for training and validation and many more subjects for testing, 18.