Semi-Supervised Learning with Sparse Autoencodersin Phone Classification
Table 1: Results on frame-based phoneme classification on the validation and test sets on the TIMIT material. Our method (SSSAE) is compared to a neural network trained with supervised backpropagation with the same amount of labelled data. The total number of training frames is 1068818. The value of α is optimised on the validation set as the proportion of labelled examples is varied.
['Results on TIMIT Labelled Observations', 'Results on TIMIT Labelled Observations', 'Results on TIMIT Neural Network', 'Results on TIMIT Neural Network', 'Results on TIMIT SSSAE', 'Results on TIMIT SSSAE', 'Results on TIMIT']
NN: neural network trained with supervised backpropagation. SSSAE: our method. Both validation and test accuracy rates are shown. Both validation set accuracy and test set accuracy are reported. The hyper-parameters of the neural network such as learning rate tuned using the validation set are also shown in the table. As expected this advantage decreases when the proportion of labelled training examples is increased. The validation and test errors are always very close, indicating that the parameters optimised on the validation set generalise well to the test set. As expected, the optimal value for α is strongly dependent on the proportion of labelled material. The higher the proportion the more weight the algorithm gives to the classification error, compared to the unsupervised reconstruction error.