Multilingual Bottleneck Features for Query by Example Spoken Term Detection
Table 1: Performance of the QbE-STD system in QUESST 2014 database using various monolingual and multilingual bottleneck features for different types of evaluation queries. Cminnxe (lower is better) and MTWV (higher is better) is used as evaluation metric.
['Monolingual Feature', 'Training Language', 'System', 'T1 Queries  [ITALIC] Cmin [ITALIC] nxe↓', 'T1 Queries  [ITALIC] MTWV↑', 'T2 Queries  [ITALIC] Cmin [ITALIC] nxe↓', 'T2 Queries  [ITALIC] MTWV↑', 'T3 Queries  [ITALIC] Cmin [ITALIC] nxe↓', 'T3 Queries  [ITALIC] MTWV↑']
We train five different monolingual networks for both architectures: FFN and ResNet, corresponding to PT, ES, RU, FR, GE languages from GlobalPhone database. We can see that the ResNet based bottleneck features perform better than most of the FFN based features in terms of Cminnxe metric, except for T3 queries with FR, ES and RU features, where the performances are close. We also observe that PT features perform best for both FFN and ResNet. We present the results of our multitask learning based multilingual systems and compare their performance with a simple monolingual feature concatenation approach. [leftmargin=0.3cm] Multitask Learning: The 3 language network uses the best performing monolingual training languages. Clearly, ResNet based bottleneck features provide significant improvement over the corresponding FFN based features. We also observe that PT-ES-RU-FR-GE features significantly outperform PT-ES-RU features for both FFN and ResNet model indicating that additional languages for training provide better language independent features. Feature Concatenation: Another way of utilizing training resources from multiple languages is to concatenate the monolingual bottleneck features to perform DTW. We perform two sets of experiments by concatenating monolingual features from PT-ES-RU and FR-GE-PT-ES-RU languages corresponding to both FFN and ResNet. On the other hand, ResNet based feature (PT-ES-RU) perform significantly better than the corresponding PT features. However, there is no significant performance difference between the ResNet based 3 and 5 language feature concatenation.