A Survey of Recent DNN Architectures on the TIMIT Phone Recognition Task This paper was supported by the project no. P103/12/G084 of the Grant Agency of the Czech Republic and by the grant of the University of West Bohemia, project No. SGS-2016-039. Access to computing and storage facilities owned by parties and projects contributing to the National Grid Infrastructure MetaCentrum provided under the programme "Projects of Large Research, Development, and Innovations Infrastructures" (CESNET LM2015042), is greatly appreciated.
Table 1: Feed-Forward DNN Phone Error Rate
['Network', 'Development PER Min', 'Development PER Max', 'Development PER Avg', 'Test PER Min', 'Test PER Max', 'Test PER Avg']
After we trained all the networks, we have evaluated their performance on the development and test dataset. The FF network with sigmoid activation function and DBN pre-training has average 17.04 % PER, but it was outperformed by several simple FF networks with ReLU without pretraining. The best average PER we have achieved is 16.91 % PER in the case of 8x2048 FF network. However, all the deeper networks with at least 1024 neurons in the hidden layers have similar performance.