CSTNet: Contrastive Speech Translation Network for Self-Supervised Speech Representation Learning
Table 4: Phone Recognition Results with features extracted from different layers of CSTNet on WSJ eval92. LX stands for Layer #X.
['[BOLD] Language pair', '[BOLD] PER L5', '[BOLD] PER L6', '[BOLD] PER L7', '[BOLD] PER L8', '[BOLD] PER L9', '[BOLD] PER L10', '[BOLD] PER L11', '[BOLD] PER L12', '[BOLD] PER L13']
Rows in the table correspond to the CSTNet trained on different language pairs and the different columns correspond to the layer number of the pre-trained audio network. As can be seen from the table, as we go higher up in the audio network, the phone recognition performance improves with the best performance achieved using the representations extracted from the last layer. The improvement in PER is not consistent with the increase in layer size but the best performance is always from last layer. This is not surprising as the last layer has the highest receptive field and hence has access to more global information essential for the task of phone recognition. The best performance is obtained using the CSTNet trained on the language pair English-French, but the performance gap between different language pairs is not very significant, with a gap of 2.8 percentage points between the best (en-fr) and the worst (en-de) language pairs.